r/LocalLLaMA
· Communities
LFM2.5-2.6B model+KV cache quantization report
LFM2.5-2.6B is a new tiny model by LiquidAI, with benchmarks that put it head to head with much larger models. I've run llama-perplexity on many model GGUF quants, crossed with many KV cache quants, to understand the model's best overall quantization for any given amount of memory. I also show how different quantizatio