HF Daily Papers
· Papers
Skaling: Chinchilla's Exponents Meet Kaplan's Coupling
Neural scaling laws are foundational for language model development, yet standard formulations systematically under- and overestimate loss at data-scarce and overtraining extremes. This failure originates in the underlying assumption that model size and training data impact the loss independently. T