arXiv stat.ML
· Papers
Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime
arXiv:2603.10485v3 Announce Type: replace Abstract: In this work, we study the convergence properties of the Dual Space Preconditioned Gradient Descent, encompassing optimizers such as Normalized Gradient Descent and Gradient Clipping. We consider preconditioners of the form $nabla K$, where $K: mathbb{R}^{d times k