arXiv cs.LG
· Papers
Anti-Collapse Dynamics and the Emergence of Multi-Time-Scale Learning in Recurrent Neural Networks
arXiv:2606.29519v2 Announce Type: replace Abstract: Long-range learning is hard for recurrent networks trained with stochastic gradient descent, because the influence of a past input fades with the lag $ell$, and if it fades too fast the dependence cannot be learned from finite data. This fade is captured by an envelo