arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.01635math.NAcs.DScs.LGcs.NAmath.OC
收藏

广义Stiefel流形上具有预处理格式的黎曼optimization

Riemannian optimization with a preconditioning scheme on the generalized Stiefel manifold

Tel Aviv University(特拉维夫大学)

Boris Shustin, Haim Avron

摘要:

广义Stiefel流形(及其乘积)上的最优化问题在科学和工程中是普遍存在的。例如,在计算科学中,它们出现在对称(广义)本征值问题、非线性本征值问题和电子结构计算中,仅举几个问题。在统计学和机器学习中,它们出现在各种降维技术中,例如典型相关分析。在深度学习中,正则化和改进的稳定性可以通过将某些层约束为具有属于Stiefel流形的参数矩阵来获得。广义Stiefel流形上的问题可以通过黎曼优化的工具来解决。然而,使用标准的几何组件的广义Stiefel流形有两个可能的缺点:计算一些几何组件可能过于昂贵,在某些情况下收敛可能会相当慢。这两个缺点都可以使用称为黎曼预处理的技术来解决,该技术相当于使用由在约束流形上定义黎曼度量的预处理器导出的几何分量。在本文中,我们开发的几何组件进行黎曼优化的广义Stiefel流形上配备了一个非标准的度量,并从理论和数值上说明使用这些组件和黎曼预处理的效果,解决最优化问题的广义Stiefel流形。

英文摘要:

Optimization problems on the generalized Stiefel manifold (and products of it) are prevalent across science and engineering. For example, in computational science they arise in symmetric (generalized) eigenvalue problems, in nonlinear eigenvalue problems, and in electronic structures computations, to name a few problems. In statistics and machine learning, they arise, for example, in various dimensionality reduction techniques such as canonical correlation analysis. In deep learning, regularization and improved stability can be obtained by constraining some layers to have parameter matrices that belong to the Stiefel manifold. Solving problems on the generalized Stiefel manifold can be approached via the tools of Riemannian optimization. However, using the standard geometric components for the generalized Stiefel manifold has two possible shortcomings: computing some of the geometric components can be too expensive and convergence can be rather slow in certain cases. Both shortcomings can be addressed using a technique called Riemannian preconditioning, which amounts to using geometric components derived by a precoditioner that defines a Riemannian metric on the constraint manifold. In this paper we develop the geometric components required to perform Riemannian optimization on the generalized Stiefel manifold equipped with a non-standard metric, and illustrate theoretically and numerically the use of those components and the effect of Riemannian preconditioning for solving optimization problems on the generalized Stiefel manifold.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑