RT Artur ChakhvadzeRoPE is equivalent to KDA with imaginary log-decay factorsALiBi is KDA with real log-decay factorsState-space models don’t need positional embeddings because positional embeddings are…
Kimi K3 is a triumph of scaling maximalism. They scaled the hell out of everything.
reminder that my method for forecasting is literally just greedy decodingTeortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞): 83% would be very very good. my ranges are 0.7-0.75…
Damn, Xi Jinping's sleeping pills are something fierce, must be the power of traditional Chinese medicine…according to Fable, Kimi K3 is roughly in the ballpark of…
RT Brian McDonaldNoah Smith has built a paid newsletter partly on the proposition that his analysis is worth more than ordinary internet commentary.Which makes it fairly…
Local guys hardest hitThese models are built for datacenters You would have better luck distilling from themEric: KIMI-K3 PRISM-DQ (Dynamic Quant) Recipe Analysis is out.I know…
RT He-Man, Principal MLE"Let's all post a formal outrage response on the same platform with largely known associated accounts, this will surely read as legitimate grassroots…
If this proves to be true, we are in a much more interesting timeline than currently believedLeroy Li: 5 units this year actually represent the second…
RT Susan Zhangopen-source is beautiful, indeedKimi.ai: We've open-sourced MoonEP, our high-performance communication library for distributed MoE workloads. Built to make expert-parallel communication more efficient at scale,…
I’m not sure if bullying a GDM policy staff on a paper-thin pretext of “we’re mentally ill/boomers and don’t get memes” is a sound strategy of…