AI 中文总结
本文提出一种Stiefel流形上带一次Newton-Schulz迭代的不精确黎曼梯度下降算法，可保证收敛性，性能优于同类算法，还提出其随机版本且表现优异。
AI 中文摘要
本文提出了一种在Stiefel流形上的不精确黎曼梯度下降算法（IRGS-StieONS），采用自适应步长，其中“不精确”指的是收缩（retraction）的不精确性。研究证明，收缩的单次Newton-Schulz迭代足以保证全局收敛性和局部线性收敛性。与基于着陆和增广拉格朗日的算法相比，所提算法是首个允许采用实用初始步长的自适应步长的不可行算法，且在温和假设下可保证全局收敛性和局部线性收敛性。此外，研究表明局部收敛速率取决于黎曼海森矩阵的条件数，这与黎曼最速下降算法一致，该结果意味着所提算法中的不可行性不影响局部收敛速率。进一步地，提出了IRGD-StieONS的随机梯度版本，其收敛速率与采用递减步长的黎曼随机梯度下降算法相同。数值实验表明，IRGD-StieONS及其随机对应版本均表现出优异的性能和鲁棒性。
英文摘要
In this paper, we propose an inexact Riemannian gradient descent algorithm on the Stiefel manifold (IRGS-StieONS) using an adaptive step size, where the ``inexact'' refers to the inexactness of retraction. It is proven that one single Newton-Schulz iteration for the retraction is sufficient for global convergence and local linear convergence. Compared to the landing and augmented Lagrangian-based algorithms, the proposed algorithm is the first infeasible algorithm that permits adaptive step sizes with a practical initial step size and guarantees global convergence and local linear convergence under mild assumptions. Moreover, we show that the local convergence rate depends on the condition number of the Riemannian Hessian, which matches the Riemannian steepest descent algorithm. This result implies that the infeasibility in the proposed algorithm does not influence the local convergence rate. Furthermore, a stochastic gradient version of IRGD-StieONS is proposed and is shown to achieve the same convergence rate as Riemannian stochastic gradient descent with decreasing step size. Numerical experiments demonstrate that both IRGD-StieONS and its stochastic counterpart exhibit superior performance and robustness.
