r/LocalLLaMA
· Communities
What KV cache tricks do you guys do with your agents?
(Typing on my phone, apologies in advance for my “shorthand”.) I recently learned about prompt caching in llama.cpp. Basically, it’s a setting where your kv cache is stored in RAM as a copy and ejected on system prompt delta via idle slot setting to make room for KV cache swap. I use it for running subagents (or sequen