HF Daily Papers
· Papers
Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing
Sparse mixture-of-experts (MoE) language models route each token to multiple experts, suggesting a geometric account of their benefit: co-selected experts should contribute distinct representation directions. Existing evidence often conflates route coherence, candidate quality, and candidate-by-cont