r/LocalLLaMA
· Communities
[Paper] xHC: Expanded Hyper-Connections – Scale Residual Streams Wider · Push Model Intelligence Further
Hyper-Connections (HC) expand the residual stream of Transformers into N parallel streams, providing a form of memory scaling beyond model width and depth. Manifold-Constrained HC (mHC) stabilizes this formulation at scale. The large gains from N{=}1 to N{=}4 suggest residual-stream expansion as a promising scaling axi