X · @teortaxesTex
· X / Twitter
It's important that not only has the Chinese ecosystem coalesced around MoE with wide expert parallelism to survive on weaker NPUs, but hardware devel…
It's important that not only has the Chinese ecosystem coalesced around MoE with wide expert parallelism to survive on weaker NPUs, but hardware developers are also designing systems with large domains. Everyone's got to have a "supernode" now. It's not just Huawei.GDP: The Chinese ecosystem has coalesced around MoE ar