arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.05755math.OCcs.SYeess.SY
收藏

分散梯度下降算法步长的一个紧界

A tight bound on the stepsize of the decentralized gradient descent

Woocheol Choi

摘要:

本文研究了由\begin{equation*} x_i (t+1) = \sum_{j=1}^m w_{ij} x_j (t) - α(t) \nabla f_i (x_i (t)). \end{equation*}给出的分散梯度网下降(DGD)问题。当总代价$f$是强凸的且局部代价可能是非凸的时,我们发现了一个步长$α(t)>0$的尖锐范围,使得序列$\{x_i (t)\}$是一致有界的。精确地说,我们找到了一个紧界$α_0 >0$,使得对于满足$α(0) \leq α_0$的非增序列$α(t)$,DGD算法的状态是一致有界的。数值实验也验证了理论结果。

英文摘要:

In this paper, we consider the decentralized gradinet descent (DGD) given by \begin{equation*} x_i (t+1) = \sum_{j=1}^m w_{ij} x_j (t) - α(t) \nabla f_i (x_i (t)). \end{equation*} We find a sharp range of the stepsize $α(t)>0$ such that the sequence $\{x_i (t)\}$ is uniformly bounded when the aggregate cost $f$ is assumed be strongly convex with smooth local costs which might be non-convex. Precisely, we find a tight bound $α_0 >0$ such that the states of the DGD algorithm is uniformly bounded for non-increasing sequence $α(t)$ satisfying $α(0) \leq α_0$. The theoretical results are also verified by numerical experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑