一阶与零阶去中心化非光滑非凸随机优化的在线优化视角
Online Optimization Perspective on First-Order and Zero-Order Decentralized Nonsmooth Nonconvex Stochastic Optimization
浏览论文内容
中文总结 AI 辅助
研究去中心化非光滑非凸随机优化中寻找稳定点的问题，提出ME-DOL算法，利用在线优化技术在一阶和零阶设置下实现了与最优集中式方案匹配的样本复杂度。
中文摘要 AI 辅助
我们研究了在去中心化随机优化中，寻找非光滑非凸目标的($δ,ε$)-稳定点的有限时间分析。一组智能体旨在仅通过其局部信息在网络交互来最小化一个全局函数。我们提出了一种名为Multi Epoch Decentralized Online Learning (ME-DOL)的新算法，并确立了其在各种设置下的样本复杂度。首先，利用最近提出的一种online-to-nonconvex技术，我们证明了该算法能恢复光滑非凸目标的最优收敛速率。接着，基于随机平滑和Goldstein-次微分集的性质，我们将分析扩展至非光滑设置。我们确立了$O(δ^{-1}ε^{-3})$的样本复杂度，据我们所知，这是一阶设置下（无弱凸性）去中心化非光滑非凸随机优化的首个有限时间保证，且与其最优的集中式对应方案相匹配。我们进一步在不使用方差缩减的情况下，为零阶预言机设置证明了相同的速率。
英文摘要
We investigate the finite-time analysis of finding ($δ,ε$)-stationary points for nonsmooth nonconvex objectives in decentralized stochastic optimization. A set of agents aim at minimizing a global function using only their local information by interacting over a network. We present a novel algorithm, called Multi Epoch Decentralized Online Learning (ME-DOL), for which we establish the sample complexity in various settings. First, using a recently proposed online-to-nonconvex technique, we show that our algorithm recovers the optimal convergence rate of smooth nonconvex objectives. We then extend our analysis to the nonsmooth setting, building on properties of randomized smoothing and Goldstein-subdifferential sets. We establish the sample complexity of $O(δ^{-1}ε^{-3})$, which to the best of our knowledge is the first finite-time guarantee for decentralized nonsmooth nonconvex stochastic optimization in the first-order setting (without weak-convexity), matching its optimal centralized counterpart. We further prove the same rate for the zero-order oracle setting without using variance reduction.
发表机构
- Northeastern University（东北大学）
机构由 AI 辅助整理，请以论文原文为准。
