X · @teortaxesTex
· X / Twitter
2800/896*16 = 50 ✅ 1600/384*7= 29,166 ❌ Bloomberg should learn arithmetic. Activated parameters cannot be calculated from expert sparsity ratio alon…
2800/896*16 = 50 ✅1600/384*7= 29,166 ❌Bloomberg should learn arithmetic. Activated parameters cannot be calculated from expert sparsity ratio alone, at a minimum there's the attention backbone. K3 is sparse but it's not 50/2800 sparse.tae kim: Kimi K3 requires a lot of memory for all those 2.8 trillion parameters (Bloo