arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2110.05282math.OC
收藏

分散优化的最优梯度跟踪

Optimal Gradient Tracking for Decentralized Optimization

Zhuoqing Song, Lei Shi, Shi Pu, Ming Yan

备注：
Mathematical Programming, in press
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出最优梯度跟踪（OGT）方法，首次在分散优化中同时达到最优梯度计算和通信复杂度，并引入快照梯度跟踪与无环切比雪夫加速作为通用构建模块。

摘要：

本文研究在多智能体网络上最小化$n$个目标函数之和的分散优化问题。智能体嵌入在无向图中，只能直接与邻居交换信息。假设目标函数光滑且强凸，我们提出一种最优梯度跟踪（OGT）方法，同时达到最优的梯度计算复杂度$O\left(\sqrtκ\log\frac{1}ε\right)$和最优的通信复杂度$O\left(\sqrt{\fracκθ}\log\frac{1}ε\right)$，其中$κ$和$\frac{1}θ$分别表示与目标函数和通信图相关的条件数。据我们所知，OGT是第一个在梯度计算和通信复杂度上均最优的单循环分散梯度型方法。OGT的开发涉及两个具有独立价值的构建模块。第一个是另一种新的分散梯度跟踪方法，称为“快照”梯度跟踪（SS-GT），其梯度计算和通信复杂度分别为$O\left(\sqrtκ\log\frac{1}ε\right)$和$O\left(\frac{\sqrtκ}θ\log\frac{1}ε\right)$。与OGT相比，SS-GT可能扩展到更一般的设置。第二个是一种称为无环切比雪夫加速（LCA）的技术，可以“无环”地实现，但能达到与在算法中添加多个内环切比雪夫加速类似的效果。除了SS-GT，该LCA技术还可以加速许多其他基于梯度跟踪的方法，相对于图条件数$\frac{1}θ$。

英文摘要：

In this paper, we focus on solving the decentralized optimization problem of minimizing the sum of $n$ objective functions over a multi-agent network. The agents are embedded in an undirected graph where they can only send/receive information directly to/from their immediate neighbors. Assuming smooth and strongly convex objective functions, we propose an Optimal Gradient Tracking (OGT) method that achieves the optimal gradient computation complexity $O\left(\sqrtκ\log\frac{1}ε\right)$ and the optimal communication complexity $O\left(\sqrt{\fracκθ}\log\frac{1}ε\right)$ simultaneously, where $κ$ and $\frac{1}θ$ denote the condition numbers related to the objective functions and the communication graph, respectively. To our knowledge, OGT is the first single-loop decentralized gradient-type method that is optimal in both gradient computation and communication complexities. The development of OGT involves two building blocks which are also of independent interest. The first one is another new decentralized gradient tracking method termed "Snapshot" Gradient Tracking (SS-GT), which achieves the gradient computation and communication complexities of $O\left(\sqrtκ\log\frac{1}ε\right)$ and $O\left(\frac{\sqrtκ}θ\log\frac{1}ε\right)$, respectively. SS-GT can be potentially extended to more general settings compared to OGT. The second one is a technique termed Loopless Chebyshev Acceleration (LCA) which can be implemented "looplessly" but achieve similar effect with adding multiple inner loops of Chebyshev acceleration in the algorithms. In addition to SS-GT, this LCA technique can accelerate many other gradient tracking based methods with respect to the graph condition number $\frac{1}θ$.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑