一种黎曼AdaGrad-Norm方法
A Riemannian AdaGrad-Norm Method
AI总结:
提出黎曼AdaGrad-Norm方法(MAdaGrad),每次迭代仅需一次指数映射,在多种假设下给出收敛复杂度界,并在对称正定矩阵流形上构造非凸PL函数,数值实验验证其优于Armijo线搜索的黎曼最速下降。
摘要:
我们提出了一种流形上的AdaGrad-Norm方法(\textsc{MAdaGrad}),该方法将AdaGrad的范数版本(AdaGrad-Norm)扩展到黎曼优化中。与每次迭代可能需要多次指数映射计算的线搜索方案相比,\textsc{MAdaGrad}每次迭代仅需一次指数映射计算。假设目标函数$f$具有Lipschitz连续的黎曼梯度,我们证明该方法最多需要$\mathcal{O}(\varepsilon^{-2})$次迭代即可计算出一个满足$\\|\operatorname{grad} f(x)\\||\leq \varepsilon$的点$x$。在额外假设$f$是测地凸且流形的截面曲率有下界的条件下,我们证明该方法最多需要$\mathcal{O}(\varepsilon^{-1})$次迭代即可找到满足$f(x)-f_{low}\leq\varepsilon$的$x$,其中$f_{low}$是最优值。此外,如果$f$在流形上全局满足Polyak--Łojasiewicz条件,我们建立了$\mathcal{O}(\log(\varepsilon^{-1}))$的复杂度界,前提是初始黎曼梯度的范数足够大。对于对称正定矩阵流形,我们构造了一族满足PL条件的非凸函数。数值实验展示了\textsc{MAdaGrad}与配备Armijo线搜索的黎曼最速下降法相比的显著性能。
英文摘要:
We propose a manifold AdaGrad-Norm method (\textsc{MAdaGrad}), which extends the norm version of AdaGrad (AdaGrad-Norm) to Riemannian optimization. In contrast to line-search schemes, which may require several exponential map computations per iteration, \textsc{MAdaGrad} requires only one. Assuming the objective function $f$ has Lipschitz continuous Riemannian gradient, we show that the method requires at most $\mathcal{O}(\varepsilon^{-2})$ iterations to compute a point $x$ such that $\|\operatorname{grad} f(x)\|\leq \varepsilon$. Under the additional assumptions that $f$ is geodesically convex and the manifold has sectional curvature bounded from below, we show that the method takes at most $\mathcal{O}(\varepsilon^{-1})$ to find $x$ such that $f(x)-f_{low}\leqε$, where $f_{low}$ is the optimal value. Moreover, if $f$ satisfies the Polyak--Łojasiewicz condition globally on the manifold, we establish a complexity bound of $\mathcal{O}(\log(\varepsilon^{-1}))$, provided that the norm of the initial Riemannian gradient is sufficiently large. For the manifold of symmetric positive definite matrices, we construct a family of nonconvex functions satisfying the PL condition. Numerical experiments illustrate the remarkable performance of \textsc{MAdaGrad} in comparison with Riemannian Steepest Descent equipped with Armijo line-search.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。