预算约束下的多共识分布式梯度下降

中文摘要 AI 辅助

我们研究了分布式梯度下降（DGD），重点是在预算约束下高效利用通信和计算资源。作为更广泛的通信-计算分配问题的第一步，我们考虑并分析了一种多共识分布式梯度下降（mcDGD）方案，其中共识轮数和步长允许在不同迭代之间变化。基于DGD的统一分析框架，我们推导了有限时间收敛界，该界明确刻画了共识质量与优化动态之间的相互作用。我们的分析仅要求局部目标函数具有凸性，同时假设全局目标函数具有光滑性和强凸性。所得界使得在资源约束下能够采用有原则的共识分配策略，我们证明了在我们的步长规则下，跨迭代的共识轮数均等分配在整数舍入范围内是最优的。数值实验证实了理论发现，并展示了与现有多种共识分布式优化基线相比，在通信-计算权衡方面的优越性。

英文摘要

We investigate decentralized gradient descent (DGD) with emphasis on efficient communication and computation resource utilization under budget constraints. As a first step toward the broader communication-computation allocation problem, we consider and analyze a \textit{multi-consensus decentralized gradient descent} (mcDGD) scheme, where the number of consensus rounds and the stepsize are allowed to vary across iterations. Building on a unified analytical framework for DGD, we derive finite-time convergence bounds that explicitly characterize the interaction between consensus quality and optimization dynamics. Our analysis requires only convexity of the local objective functions while assuming smoothness and strong convexity of the global objective. The resulting bounds enable a principled consensus-allocation strategy under resource constraints, for which we show that equal allocation of consensus rounds across iterations is optimal under our stepsize rule, up to integer rounding. Numerical experiments corroborate the theoretical findings and demonstrate favorable communication-computation tradeoffs compared with existing multi-consensus decentralized optimization baselines.

发表机构

• Linköping University（林雪平大学）
• Arizona State University（亚利桑那州立大学）

机构由 AI 辅助整理，请以论文原文为准。