arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.10070eess.SYcs.MAcs.SY
收藏

有向图上线性收敛的分布式优化算法

Gradient-Consensus: Linearly Convergent Distributed Optimization Algorithm over Directed Graphs

Vivek Khatana, Govind Saraswat, Sourav Patel, Murti V. Salapaka

摘要:

在这篇文章中,我们提出了一种新的方法,优化然后同意最小化的总和$ f = \sum_{i=1}^n f_i(x)$凸目标函数在有向图。优化然后同意方法在分布式优化框架中将优化步骤和共识步骤合并。优化然后同意的关键动机是保证在分布式优化算法的每次迭代期间代理的估计之间的分歧保持在任何先验指定的公差下;现有算法不提供在许多实际场景中所需的这样的保证。在这种方法中,每个代理在每次迭代期间保持对最优解的估计,并利用其局部可用的梯度信息以及有限时间近似共识协议来向最优解移动(因此称为一致性共识算法)。我们建立了所提出的算法具有全局R-线性收敛速度,如果聚合函数$f$是强凸和Lipschitz可微。在放松的$f_i$是凸的和Lipschitz可微的假设下,我们还证明了目标函数误差残差以Q-线性速率减小(就梯度计算步骤的数量而言),直到它达到一个小值,这可以使用有限时间近似共识协议上指定的容差值来管理;当$f_i$不一定是强凸函数时,文献中没有现有的方法具有如此强的收敛保证。与传统算法的$O(k)$相比,改进的算法在满足约束条件和更好收敛性的保证方面的通信开销是$O(k\log k)$迭代次数。此外,我们数值评估所提出的算法的性能,通过解决分布式逻辑回归问题。

英文摘要:

In this article, we propose a new approach, optimize then agree for minimizing a sum $ f = \sum_{i=1}^n f_i(x)$ of convex objective functions over a directed graph. The optimize then agree approach decouples the optimization step and the consensus step in a distributed optimization framework. The key motivation for optimize then agree is to guarantee that the disagreement between the estimates of the agents during every iteration of the distributed optimization algorithm remains under any apriori specified tolerance; existing algorithms do not provide such a guarantee which is required in many practical scenarios. In this method, each agent during each iteration maintains an estimate of the optimal solution and, utilizes its locally available gradient information along with a finite-time approximate consensus protocol to move towards the optimal solution (hence the name Gradient-Consensus algorithm). We establish that the proposed algorithm has a global R-linear rate of convergence if the aggregate function $f$ is strongly convex and Lipschitz differentiable. We also show that under the relaxed assumption of $f_i$'s being convex and Lipschitz differentiable, the objective function error residual decreases at a Q-linear rate (in terms of the number of gradient computation steps) until it reaches a small value, which can be managed using the tolerance value specified on the finite-time approximate consensus protocol; no existing method in the literature has such strong convergence guarantees when $f_i$ are not necessarily strongly convex functions. The communication overhead for the improved guarantees on meeting constraints and better convergence of our algorithm is $O(k\log k)$ iterates in comparison to $O(k)$ of the traditional algorithms. Further, we numerically evaluate the performance of the proposed algorithm by solving a distributed logistic regression problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑