arXiv cs.LG
· Papers
QEvict: Recoverable Quantized KV Eviction for Attention-Drift-Robust Long-Context Decoding
arXiv:2608.05326v1 Announce Type: new Abstract: Autoregressive large language model inference is increasingly constrained by the memory footprint of the Key-Value (KV) cache. A dominant line of work reduces this footprint by evicting tokens that appear unimportant under attention-derived scores. However, such policies