r/LocalLLaMA
· Communities
How to even compare quants from various sources?
How do you guys deal with so many variables? So many publishers, each calling their quants "best", and then it's a mess to manage, download the weights, tweak the temperature etc. for each source? It is relatively simple if I'm comparing different quantization levels (like Q4 vs Q5), that's mostly linear and I run the