r/LocalLLaMA
· Communities
Laguna S 2.1 GGUF Q4_K_M went from 68GB to 96GB?
I've been occasionally checking Laguna S 2.1 to see if there's any updates/fixes to the issues they've been having. I just noticed that they recently updated their Q4_K_M and it's now ballooned to 96GB, bumping up 8 layers to FP16 while leaving the rest in 4-bit. Does anyone know why they would do this? I'm guessing be