X · @teortaxesTex
· X / Twitter
RT Zitian Gao: Re Hello, thank you for your question. I am the first author of this paper. We spent hundres of millions of USD of compute validating t…
RT Zitian GaoRe Hello, thank you for your question. I am the first author of this paper.We spent hundres of millions of USD of compute validating the classical Universal Transformer, which we refer to as the “model-loop.” However, we found that it does not scale well to MoE models with billions of parameters.Please rea