arXiv cs.NE
· Papers
RotaryQuant: Fitting 120B MoE Models on Consumer Hardware via Fused Compressed-Space Attention
arXiv:2608.08081v1 Announce Type: new Abstract: Large mixture-of-experts (MoE) language models with 26--120 billion parameters exceed the memory capacity of consumer devices through three simultaneous pressures: resident weight matrices, key-value (KV) cache state that grows linearly with context, and dozens of expert