arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.04026math.OCcs.LGstat.ML
收藏

非光滑与低秩矩阵优化问题的低秩额外梯度法

Low-Rank Extragradient Method for Nonsmooth and Low-Rank Matrix Optimization Problems

Technion - Israel Institute of Technology(以色列理工学院)

Dan Garber, Atara Kaplan

AI总结:

本文针对非光滑低秩矩阵优化问题,在广义严格互补条件下,证明了使用热启动的额外梯度法每次仅需两次低秩SVD即可收敛至最优解,且与全秩SVD产生相同迭代序列。

摘要:

低秩和非光滑矩阵优化问题涵盖了统计学和机器学习中的许多基本任务。尽管近年来在开发避免维护高秩矩阵和计算昂贵的高秩SVD的高效方法来解决光滑低秩优化问题方面取得了重大进展,但针对非光滑问题的进展却十分缓慢。在本文中,我们考虑此类问题的标准凸松弛。主要地,我们证明了在自然的广义严格互补条件以及非光滑目标可以表示为光滑函数最大值的相对温和假设下,当使用“热启动”点进行初始化时,额外梯度法能以O(1/t)的速率收敛至最优解,且每次迭代仅需两次低秩SVD。我们给出了所需的SVD秩与我们需初始化该方法的球的半径之间的精确权衡。我们通过在多个非光滑低秩矩阵恢复任务上的实证实验支持了我们的理论结果,证明了使用简单的初始化,当用与待恢复的(低秩)真实矩阵秩相匹配的秩的SVD替换全秩SVD时,额外梯度法能产生完全相同的迭代序列。

英文摘要:

Low-rank and nonsmooth matrix optimization problems capture many fundamental tasks in statistics and machine learning. While significant progress has been made in recent years in developing efficient methods for \textit{smooth} low-rank optimization problems that avoid maintaining high-rank matrices and computing expensive high-rank SVDs, advances for nonsmooth problems have been slow paced. In this paper we consider standard convex relaxations for such problems. Mainly, we prove that under a natural \textit{generalized strict complementarity} condition and under the relatively mild assumption that the nonsmooth objective can be written as a maximum of smooth functions, the \textit{extragradient method}, when initialized with a ``warm-start'' point, converges to an optimal solution with rate $O(1/t)$ while requiring only two \textit{low-rank} SVDs per iteration. We give a precise trade-off between the rank of the SVDs required and the radius of the ball in which we need to initialize the method. We support our theoretical results with empirical experiments on several nonsmooth low-rank matrix recovery tasks, demonstrating that using simple initializations, the extragradient method produces exactly the same iterates when full-rank SVDs are replaced with SVDs of rank that matches the rank of the (low-rank) ground-truth matrix to be recovered.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑