arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2509.19682math.OC
收藏

一种黎曼AdaGrad-Norm方法

A Riemannian AdaGrad-Norm Method

Nankai University(南开大学)

Glaydston de C. Bento, Geovani N. Grapiglia, Mauricio S. Louzeiro, Daoping Zhang

AI总结:

提出黎曼AdaGrad-Norm方法(MAdaGrad),每次迭代仅需一次指数映射,在多种假设下给出收敛复杂度界,并在对称正定矩阵流形上构造非凸PL函数,数值实验验证其优于Armijo线搜索的黎曼最速下降。

摘要:

我们提出了一种流形上的AdaGrad-Norm方法(\textsc{MAdaGrad}),该方法将AdaGrad的范数版本(AdaGrad-Norm)扩展到黎曼优化中。与每次迭代可能需要多次指数映射计算的线搜索方案相比,\textsc{MAdaGrad}每次迭代仅需一次指数映射计算。假设目标函数$f$具有Lipschitz连续的黎曼梯度,我们证明该方法最多需要$\mathcal{O}(\varepsilon^{-2})$次迭代即可计算出一个满足$\\|\operatorname{grad} f(x)\\||\leq \varepsilon$的点$x$。在额外假设$f$是测地凸且流形的截面曲率有下界的条件下,我们证明该方法最多需要$\mathcal{O}(\varepsilon^{-1})$次迭代即可找到满足$f(x)-f_{low}\leq\varepsilon$的$x$,其中$f_{low}$是最优值。此外,如果$f$在流形上全局满足Polyak--Łojasiewicz条件,我们建立了$\mathcal{O}(\log(\varepsilon^{-1}))$的复杂度界,前提是初始黎曼梯度的范数足够大。对于对称正定矩阵流形,我们构造了一族满足PL条件的非凸函数。数值实验展示了\textsc{MAdaGrad}与配备Armijo线搜索的黎曼最速下降法相比的显著性能。

英文摘要:

We propose a manifold AdaGrad-Norm method (\textsc{MAdaGrad}), which extends the norm version of AdaGrad (AdaGrad-Norm) to Riemannian optimization. In contrast to line-search schemes, which may require several exponential map computations per iteration, \textsc{MAdaGrad} requires only one. Assuming the objective function $f$ has Lipschitz continuous Riemannian gradient, we show that the method requires at most $\mathcal{O}(\varepsilon^{-2})$ iterations to compute a point $x$ such that $\|\operatorname{grad} f(x)\|\leq \varepsilon$. Under the additional assumptions that $f$ is geodesically convex and the manifold has sectional curvature bounded from below, we show that the method takes at most $\mathcal{O}(\varepsilon^{-1})$ to find $x$ such that $f(x)-f_{low}\leqε$, where $f_{low}$ is the optimal value. Moreover, if $f$ satisfies the Polyak--Łojasiewicz condition globally on the manifold, we establish a complexity bound of $\mathcal{O}(\log(\varepsilon^{-1}))$, provided that the norm of the initial Riemannian gradient is sufficiently large. For the manifold of symmetric positive definite matrices, we construct a family of nonconvex functions satisfying the PL condition. Numerical experiments illustrate the remarkable performance of \textsc{MAdaGrad} in comparison with Riemannian Steepest Descent equipped with Armijo line-search.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑