r/LocalLLaMA
· Communities
To KL Diverge, or Not to KL Diverge: A Question for Quants
Hey r/LocalLLaMA! Apparently, if you draw enough arrows between proxy rankings like KLD, perplexity, and BPW, and real deployment measurements, quantization evaluation starts to look like abstract modern art. Check it out in the second figure! TL;DR: KLD and perplexity can help rank quantized models once degradation be