再论光滑分布Optimization的PDKEEPAAA
Revisiting EXTRA for Smooth Distributed Optimization
摘要：
EXTRA是一种流行的分布式优化方法，有着广泛的应用。本文将重新审视EXTRA。首先，我们给出了尖锐的复杂性分析与改进的$O\left(\left(\frac{L}μ+\frac{1}{1-σ_2(W)}\right)\log\frac{1}{ε(1-σ_2(W))}\right)$的通信和计算复杂性的$μ$ -强凸和$L$ -光滑的问题，其中$σ_2(W)$是第二大奇异值的权重矩阵$W$的EXTRA。当强凸性不存在时，我们证明了$O\left(\left(\frac{L}ε+\frac{1}{1-σ_2(W)}\right)\log\frac{1}{1-σ_2(W)}\right)$复杂性.然后，利用Catalyst框架对EXTRA进行加速，得到了强凸光滑问题的$O\left(\sqrt{\frac{L}{μ(1-σ_2(W))}}\log\frac{ L}{μ(1-σ_2(W))}\log\frac{1}ε\right)$通信和计算复杂度以及非强凸问题的$O\left(\sqrt{\frac{L}{ε(1-σ_2(W))}}\log\frac{1}{ε(1-σ_2(W))}\right)$复杂度.我们的加速EXTRA的通信复杂性仅差的$\left(\log\frac{L}{μ(1-σ_2(W))}\right)$和$\left(\log\frac{1}{ε(1-σ_2(W))}\right)$的因素，从较低的复杂性界限强凸和非强凸的问题，分别。
英文摘要：
EXTRA is a popular method for dencentralized distributed optimization and has broad applications. This paper revisits EXTRA. First, we give a sharp complexity analysis for EXTRA with the improved $O\left(\left(\frac{L}μ+\frac{1}{1-σ_2(W)}\right)\log\frac{1}{ε(1-σ_2(W))}\right)$ communication and computation complexities for $μ$-strongly convex and $L$-smooth problems, where $σ_2(W)$ is the second largest singular value of the weight matrix $W$. When the strong convexity is absent, we prove the $O\left(\left(\frac{L}ε+\frac{1}{1-σ_2(W)}\right)\log\frac{1}{1-σ_2(W)}\right)$ complexities. Then, we use the Catalyst framework to accelerate EXTRA and obtain the $O\left(\sqrt{\frac{L}{μ(1-σ_2(W))}}\log\frac{ L}{μ(1-σ_2(W))}\log\frac{1}ε\right)$ communication and computation complexities for strongly convex and smooth problems and the $O\left(\sqrt{\frac{L}{ε(1-σ_2(W))}}\log\frac{1}{ε(1-σ_2(W))}\right)$ complexities for non-strongly convex ones. Our communication complexities of the accelerated EXTRA are only worse by the factors of $\left(\log\frac{L}{μ(1-σ_2(W))}\right)$ and $\left(\log\frac{1}{ε(1-σ_2(W))}\right)$ from the lower complexity bounds for strongly convex and non-strongly convex problems, respectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
