arXiv stat.ML
· Papers
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
arXiv:2405.11667v2 Announce Type: replace-cross Abstract: Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically proving the dominance of local SGD in settings with reasonable data heterogeneity