r/LocalLLaMA
· Communities
You really should not quantize KV Cache for DeepSeek V4 Flash
I don't think anyone should quantize the KV with DS4F. I checked the the quality impact (PPL, KLD, Same TopP) for swhitching from BF16 KV to Q8 KV, and it appears significant. Very much in contrast to Qwen 397B. Here are the results for DS4F: ====== Perplexity statistics ====== Mean PPL(Q) : 5.877076 ± 0.042497 Mean PP