用于黎曼流形上非光滑复合期望优化的随机黎曼交替下降上升方法
A Stochastic Riemannian Alternating Descent Ascent Method for Nonsmooth Composite Expectation Optimization on Riemannian Manifolds
浏览论文内容
中文总结 AI 辅助
针对黎曼流形上非光滑复合期望优化问题，本文提出StoRADA-RM算法，证明其复杂度为O(ε⁻³)，并通过数值实验验证其性能。
中文摘要 AI 辅助
本文研究一类黎曼非光滑复合期望优化问题，这类问题出现在各类机器学习、信号处理和统计应用中。注意到这些问题可转化为结构化极小极大形式，我们提出一种高效算法，名为带递归动量的随机黎曼交替下降上升方法（StoRADA-RM），以求解这类问题。StoRADA-RM在每次迭代中执行一次或多次黎曼随机梯度下降步骤，随后执行一次近端梯度上升步骤。为计算黎曼随机梯度，我们提出一种无向量传输的递归动量估计器，该估计器每次迭代仅需O(1)次随机梯度评估。我们证明，StoRADA-RM在O(ε⁻³)次迭代内返回给定此类问题的ε-黎曼-随机-平稳点，同时进行O(ε⁻³)次随机一阶预言机（SFO）调用。迭代复杂度和SFO复杂度界均为该类问题文献中的最优结果，后者甚至与随机一阶算法求解光滑非凸优化的最优下界匹配。我们随后给出稀疏主成分分析和坐标无关稀疏估计的数值结果，以证明所提方法的优越性能。
英文摘要
In this paper, we consider a class of Riemannian nonsmooth composite expectation optimization problems, which arises in various machine learning, signal processing, and statistics applications. Noting that these problems admit structured minimax reformulations, we propose an efficient algorithm, named stochastic Riemannian alternating descent ascent method with recursive momentum (StoRADA-RM), to tackle them. StoRADA-RM performs one or multiple Riemannian stochastic gradient descent steps and then a proximal gradient ascent step at each iteration. To compute the Riemannian stochastic gradient, we propose a vector transport-free recursive momentum estimator that requires only $O(1)$ stochastic gradient evaluations per iteration. We prove that StoRADA-RM returns an $ε$-Riemannian-stochastic-stationary point of a given problem in the said class in $O(ε^{-3})$ iterations while making $O(ε^{-3})$ calls to a stochastic first-order oracle (SFO). Both the iteration complexity and SFO complexity bounds are the best known in the literature for the said class of problems. The latter even matches the optimal lower bound for smooth nonconvex optimization with stochastic first-order algorithms. We then present numerical results on sparse principal component analysis and coordinate-independent sparse estimation to demonstrate the superior performance of our proposed method.
发表机构
- Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
- University of Chinese Academy of Sciences（中国科学院大学）
- Nanjing Normal University（南京师范大学）
- Beijing University of Posts and Telecommunications（北京邮电大学）
- The Chinese University of Hong Kong（香港中文大学）
机构由 AI 辅助整理，请以论文原文为准。
