r/LocalLLaMA
· Communities
Qwen3.6-27B – Effect of KV quantization on KLD – Q8, Q6, Q5 (bartowski)
Lower is better - Quantization increases from right to left I recently made a post here about how I squeezed more context into a Q8 model of bartowski's Qwen3.6-27B. My reasoning was that in my (anecdotal) experience, a Q8 has been performing a lot better than a Q6 or a Q5. There were a lot of comments about quantizing