重新审视LocalSGD和SCAFFOLD：改进的收敛速率与缺失的分析
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis
AI总结：
本文在更弱的条件下重新分析了LocalSGD和SCAFFOLD的收敛性，证明LocalSGD在弱凸函数上无需强梯度相似性即可优于MbSGD，且受益于高阶相似性，而SCAFFOLD在更广的非二次函数上更快，提供了清晰的理论优势条件。
摘要：
LocalSGD和SCAFFOLD是分布式随机优化中广泛使用的方法，在机器学习、大规模数据处理和联邦学习中有众多应用。然而，严格建立它们相对于更简单方法（如小批量SGD（MbSGD））的理论优势已被证明具有挑战性，因为现有分析常常依赖于强假设、不现实的先决条件或过于受限的场景。在这项工作中，我们在各种现有或更弱的条件下重新审视了LocalSGD和SCAFFOLD的收敛性质，这些条件包括梯度相似性、Hessian相似性、弱凸性以及Hessian的Lipschitz连续性。我们的分析表明：（i）对于弱凸函数，LocalSGD在不需要更强的梯度相似性假设的情况下，相比MbSGD实现了更快的收敛；（ii）LocalSGD显著受益于高阶相似性和光滑性；（iii）对于更广泛的非二次函数类别，SCAFFOLD表现出比MbSGD更快的收敛。这些理论见解为理解LocalSGD和SCAFFOLD在何种条件下优于MbSGD提供了更清晰的认识。
英文摘要：
LocalSGD and SCAFFOLD are widely used methods in distributed stochastic optimization, with numerous applications in machine learning, large-scale data processing, and federated learning. However, rigorously establishing their theoretical advantages over simpler methods, such as minibatch SGD (MbSGD), has proven challenging, as existing analyses often rely on strong assumptions, unrealistic premises, or overly restrictive scenarios. In this work, we revisit the convergence properties of LocalSGD and SCAFFOLD under a variety of existing or weaker conditions, including gradient similarity, Hessian similarity, weak convexity, and Lipschitz continuity of the Hessian. Our analysis shows that (i) LocalSGD achieves faster convergence compared to MbSGD for weakly convex functions without requiring stronger gradient similarity assumptions; (ii) LocalSGD benefits significantly from higher-order similarity and smoothness; and (iii) SCAFFOLD demonstrates faster convergence than MbSGD for a broader class of non-quadratic functions. These theoretical insights provide a clearer understanding of the conditions under which LocalSGD and SCAFFOLD outperform MbSGD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
