arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.09261math.OCstat.ML
收藏

正交方向约束梯度法:从非线性等式约束到Stiefel流形

Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold

Sholom Schechtman, Daniil Tiapkin, Michael Muehlebach, Eric Moulines

摘要:

考虑光滑流形$\mathcal{M}$上非凸函数的极小化问题.我们提出了一种新的算法,正交方向约束梯度法(ODCGM),它只需要计算到一个向量空间的投影。ODCGM是不可行的,但是迭代不断地被拉向流形,确保ODCGM向$\mathcal{M}$收敛。ODCGM比需要计算收缩的经典方法简单得多。此外,我们表明,ODCGM表现出接近最佳的预言复杂性$\mathcal{O}(1/\varepsilon^2)$和$\mathcal{O}(1/\varepsilon^4)$在确定性和随机的情况下,分别。此外,我们建立了,在适当选择投影度量的情况下,我们的方法恢复了Ablin和Peyré(2022)的着陆算法,这是最近引入的Stiefel流形上的优化算法。因此,我们显着扩展了Ablin和Peyré(2022)的分析,在确定性和随机框架中建立了接近最优的速率。最后,我们进行了数值实验,这表明在高维设置的ODCGM的效率。

英文摘要:

We consider the problem of minimizing a non-convex function over a smooth manifold $\mathcal{M}$. We propose a novel algorithm, the Orthogonal Directions Constrained Gradient Method (ODCGM) which only requires computing a projection onto a vector space. ODCGM is infeasible but the iterates are constantly pulled towards the manifold, ensuring the convergence of ODCGM towards $\mathcal{M}$. ODCGM is much simpler to implement than the classical methods which require the computation of a retraction. Moreover, we show that ODCGM exhibits the near-optimal oracle complexities $\mathcal{O}(1/\varepsilon^2)$ and $\mathcal{O}(1/\varepsilon^4)$ in the deterministic and stochastic cases, respectively. Furthermore, we establish that, under an appropriate choice of the projection metric, our method recovers the landing algorithm of Ablin and Peyré (2022), a recently introduced algorithm for optimization over the Stiefel manifold. As a result, we significantly extend the analysis of Ablin and Peyré (2022), establishing near-optimal rates both in deterministic and stochastic frameworks. Finally, we perform numerical experiments which shows the efficiency of ODCGM in a high-dimensional setting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑