arXiv cs.NE
· Papers
Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay
arXiv:2607.21005v1 Announce Type: cross Abstract: Most explanations of training instability focus on emph{learning-rate criticality}, typically characterized by the Edge of Stability, beyond which optimization becomes unstable. We argue that, in practical deep neural network training, there is an additional and often