llama.cpp releases
· Infrastructure
b9908
server: enforce prompt cache RAM limit (#25070) Before this commit, --cache-ram was not a hard limit: The cache always kept at least one entry, even if that entry exceeded the RAM/token limits. Old entries were only evicted for the RAM/token limits after saving the new one, which could cause the cache to temporarily ex