Riemannian Momentum Tracking：在紧致子流形上基于动量的分布式优化
Riemannian Momentum Tracking: Distributed Optimization with Momentum on Compact Submanifolds
AI总结：
本文提出了一种在紧致子流形上基于动量的去中心化优化算法RMTracking，实现了比其他算法快$\frac{1}{1-β}$倍的收敛速度。
摘要：
具有动量的梯度下降法已被广泛应用于各种信号处理和机器学习任务，显示出在经验上优于标准梯度下降法的显著优势。然而，基于动量的分布式黎曼流形算法却很少被探索。在本文中，我们提出了Riemannian Momentum Tracking（RMTracking），一种在紧致子流形上具有动量的去中心化优化算法。鉴于紧致子流形的非凸性质，由有限个光滑（可能非凸）局部函数组成的目标函数在无向且连通的网络图中被各代理最小化。使用恒定步长，我们建立了Riemannian梯度平均的收敛率为$\mathcal{O}(\frac{1-β}{K})$，对于任意动量权重$β\in [0,1)$。特别是，当步长足够小时，RMTracking可以以$\mathcal{O}(\frac{1-β}{K})$的速率收敛到一个临界点。据我们所知，RMTracking是首个实现精确收敛的去中心化算法，其收敛速度比其他相关算法快$\frac{1}{1-β}$倍。最后，我们通过数值实验在特征值问题上验证了这些理论主张。
英文摘要：
Gradient descent with momentum has been widely applied in various signal processing and machine learning tasks, demonstrating a notable empirical advantage over standard gradient descent. However, momentum-based distributed Riemannian algorithms have been only scarcely explored. In this paper, we propose Riemannian Momentum Tracking (RMTracking), a decentralized optimization algorithm with momentum over a compact submanifold. Given the non-convex nature of compact submanifolds, the objective function, composed of a finite sum of smooth (possibly non-convex) local functions, is minimized across agents in an undirected and connected network graph. With a constant step-size, we establish an $\mathcal{O}(\frac{1-β}{K})$ convergence rate of the Riemannian gradient average for any momentum weight $β\in [0,1)$. Especially, RMTracking can achieve a convergence rate of $\mathcal{O}(\frac{1-β}{K})$ to a stationary point when the step-size is sufficiently small. To best of our knowledge, RMTracking is the first decentralized algorithm to achieve exact convergence that is $\frac{1}{1-β}$ times faster than other related algorithms. Finally, we verify these theoretical claims through numerical experiments on eigenvalue problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
