arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.06765math.OCcs.NAmath.NA
收藏

固定秩矩阵优化分析:商几何视角

On the analysis of optimization with fixed-rank matrices: a quotient geometric view

Shuyu Dong, Bin Gao, Wen Huang, Kyle A. Gallivan

AI总结:

该研究从商几何视角分析固定秩矩阵优化的黎曼梯度下降算法,推导其显式更新规则,证明其比欧几里得梯度下降更快且无需平衡技术,能线性收敛求解矩阵感知与补全问题,并通过数值实验验证。

摘要:

我们研究一类通过黎曼预条件设计的黎曼梯度下降(Riemannian gradient descent, RGD)算法,用于在$\u202f\uc1a0_k^{m\times n}$——即秩固定为$k$的$m\times n$实矩阵集合——上的优化问题。我们的分析基于$\u202f\uc1a0_k^{m\times n}$的商几何视角:通过矩阵分解将该集合与满列秩矩阵构成的两项乘积空间$\u202f\uc1a0_*^{m\times k}\times \u202f\uc1a0_*^{n\times k}$的商流形等同起来,我们得到了RGD算法更新规则的显式形式,这为分析其在秩约束优化中的收敛行为提供了一种新方法。我们随后推导出一些有趣的性质,反映了RGD与其他基于欧几里得几何的矩阵分解算法的区别。具体而言,我们证明RGD算法不仅比欧几里得梯度下降更快,而且不需要依赖平衡技术来保证效率,而后者则需要。我们进一步证明,在限制正定性质下,该RGD算法能够以线性收敛速率求解矩阵感知(matrix sensing)和矩阵补全(matrix completion)问题。我们还提供了矩阵感知和补全问题的数值实验来验证这些性质。

英文摘要:

We study a type of Riemannian gradient descent (RGD) algorithm, designed through Riemannian preconditioning, for optimization on $\mathcal{M}_k^{m\times n}$ -- the set of $m\times n$ real matrices with a fixed rank $k$. Our analysis is based on a quotient geometric view of $\mathcal{M}_k^{m\times n}$: by identifying this set with the quotient manifold of a two-term product space $\mathbb{R}_*^{m\times k}\times \mathbb{R}_*^{n\times k}$ of matrices with full column rank via matrix factorization, we find an explicit form for the update rule of the RGD algorithm, which leads to a novel approach to analysing their convergence behavior in rank-constrained optimization. We then deduce some interesting properties that reflect how RGD distinguishes from other matrix factorization algorithms such as those based on the Euclidean geometry. In particular, we show that the RGD algorithm is not only faster than Euclidean gradient descent but also does not rely on balancing techniques to ensure its efficiency while the latter does. We further show that this RGD algorithm is guaranteed to solve matrix sensing and matrix completion problems with linear convergence rate under the restricted positive definiteness property. Numerical experiments on matrix sensing and completion are provided to demonstrate these properties.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑