X · @teortaxesTex
· X / Twitter
RT tokenbender: the juiciest stuff i found for myself in infra is MoonEP i will try to explain briefly. so expert routing is a PITA in the sense that …
RT tokenbenderthe juiciest stuff i found for myself in infra is MoonEPi will try to explain briefly.so expert routing is a PITA in the sense that your model has several experts per layer and all are spread across all the GPUs in EP. now if a token arrives and router decides to that experts across all the GPUs have to b