r/LocalLLaMA
· Communities
I built a tool to actually test which weights matter before quantizing, instead of guessing (Qwen3.6-27B, 3 builds: Bedrock/Tightrope/Gambit)
Most quantization works like this: pick a bit depth, apply it everywhere, maybe let imatrix take a rough guess at what matters, ship it. Most don't check which specific weight groups can take a hit and which ones can't. It's convention and vibes. So I built a harness that quantizes one weight group at a time and measur