arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.14289math.OC
收藏

去中心化优化中梯度跟踪算法的通信与计算平衡

Balancing Communication and Computation in Gradient Tracking Algorithms for Decentralized Optimization

Albert S. Berahas, Raghu Bollapragada, Shagun Gupta

AI总结:

提出统一梯度跟踪方法的框架,灵活调整通信与计算步骤数量,建立理论收敛结果并量化改进,在二次函数和二元分类问题上验证性能。

摘要:

梯度跟踪方法已成为解决网络上去中心化优化问题最流行的方法之一。在此设定中,网络中的每个节点拥有全局目标函数的一部分,目标是共同优化该函数。每次迭代中,梯度跟踪方法执行两个操作(步骤):(1)计算局部梯度,(2)与网络中的局部邻居通信。这两个步骤的复杂性在不同应用中有所差异。本文提出了一个统一梯度跟踪方法的框架,该框架在通信和计算步骤数量上具有灵活性。我们为具有任意通信和计算步骤组合的算法框架建立了统一的理论收敛结果,并量化了这种灵活性带来的改进。该框架将流行梯度跟踪方法的结果作为特例恢复,并允许直接比较这些方法。最后,我们在二次函数和二元分类问题上展示了所提方法的性能。

英文摘要:

Gradient tracking methods have emerged as one of the most popular approaches for solving decentralized optimization problems over networks. In this setting, each node in the network has a portion of the global objective function, and the goal is to collectively optimize this function. At every iteration, gradient tracking methods perform two operations (steps): $(1)$ compute local gradients, and $(2)$ communicate information with local neighbors in the network. The complexity of these two steps varies across different applications. In this paper, we present a framework that unifies gradient tracking methods and is endowed with flexibility with respect to the number of communication and computation steps. We establish unified theoretical convergence results for the algorithmic framework with any composition of communication and computation steps, and quantify the improvements achieved as a result of this flexibility. The framework recovers the results of popular gradient tracking methods as special cases, and allows for a direct comparison of these methods. Finally, we illustrate the performance of the proposed methods on quadratic functions and binary classification problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑