arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.08694math.OCcs.DC
收藏

分布式梯度算法的二阶保证

Second-order Guarantees of Distributed Gradient Algorithms

Amir Daneshmand, Gesualdo Scutari, Vyacheslav Kungurtsev

摘要:

本文考虑网络上的分布光滑非凸无约束优化问题,网络模型为连通图。我们研究了分布式基于梯度的算法在严格鞍点附近的行为。具体来说,我们建立,(i)著名的分布式梯度下降(DGD)算法可能收敛到一个二阶平稳(SoS)解决方案的邻域;和(ii)最近的一类分布式算法的基础上梯度跟踪-实现也有向图-可能收敛到确切的SoS解决方案,从而避免(严格)鞍点。此外,对后一类算法建立了新的一阶临界点的收敛速度结果。

英文摘要:

We consider distributed smooth nonconvex unconstrained optimization over networks, modeled as a connected graph. We examine the behavior of distributed gradient-based algorithms near strict saddle points. Specifically, we establish that (i) the renowned Distributed Gradient Descent (DGD) algorithm likely converges to a neighborhood of a Second-order Stationary (SoS) solution; and (ii) the more recent class of distributed algorithms based on gradient tracking--implementable also over digraphs--likely converges to exact SoS solutions, thus avoiding (strict) saddle-points. Furthermore, new convergence rate results to first-order critical points is established for the latter class of algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑