r/LocalLLaMA
· Communities
Llama-Server is Throwing Away Your Perfectly Good KV Caches, and How to Fix It
Introduction I watched 2.49 GB of state restore from disk in 1.23 seconds — and then get thrown away. llama-server's slot save/restore promises exactly what long-context work on budget hardware needs: park a session on disk, bring it back later without paying the prefill tax again. And the restore itself works perfectl