r/LocalLLaMA July 20, 2026 · Communities

Would you trade speed for accuracy?

I've been working with activation aware quantization. Today I noticed that I can increase accuracy but also see a correlated decrease in tps. If you could choose a fast quantization or an accurate one, what would it be? For reference, the bench was GPQA. Model was Gemma 3 4b qat Test q_4: 27.8% accuracy, 331ms Control