arXiv stat.ML
· Papers
Attention is Just Another Name for Coupling? A Fast-Slow ODE Perspective on Hierarchical Pretraining
arXiv:2606.16730v2 Announce Type: replace Abstract: We re-interpret Transformer pretraining as a fast-slow, singularly perturbed flow along depth, with untied weights as its non-autonomous feature. The linearised dynamics is a depth-ordered product of layer maps. Along a token-homogeneous reference trajectory, the line