r/LocalLLaMA
· Communities
Papers for Stable LatentMoE and Gated MLA?
Four technologies used by Kimi K3 to make it SOTA: KimiDeltaAttention (used in Kimi Linear, essentially a more general gated delta net) AttnRes - described in Kimi's own publication: https://arxiv.org/pdf/2603.15031 Stable LatentMoE - LatentMoE was introduced by Nvidia first that allows sparser MoE (ie more experts per