X · @teortaxesTex
· X / Twitter
RT Ali Hatamizadeh: Worth noting: KDA is an extension of GDN with vector-based decay. GDN was previously proven to work at scale across many releases …
RT Ali HatamizadehWorth noting: KDA is an extension of GDN with vector-based decay. GDN was previously proven to work at scale across many releases of @Alibaba_Qwen's excellent Qwen 3.5/Qwen3-Next models.The good news: GDN-2, released a few months ago, unifies KDA and GDN by decoupling and independently controlling key