X · @teortaxesTex
· X / Twitter
beyond mHC: scaling the residual stream by effectively 16x
beyond mHC: scaling the residual stream by effectively 16xXiangdong Zhang: Hyper-Connections never fade. They evolve.To push model intelligence further, we need to scale residual streams wider and make every stream count.🚀 Today, we're introducing xHC: Expanded Hyper-Connections.HC showed that the residual stream could