arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.19477math.OC
收藏

带平衡正则化的过参数化非对称矩阵感知的因式分解梯度下降收敛速率

Convergence Rates of Factorized Gradient Descent for Overparameterized Asymmetric Matrix Sensing with Balancing Regularization

Tsinghua University(清华大学)

Yongjun Chen, Jiaming Zhang, Liping Zhang

AI总结:

针对带平衡正则化的过参数化非对称矩阵感知问题,研究因式分解梯度下降的收敛速率,通过分析其动力学子空间特性,在特定条件下证明了局部误差界,为该类问题的梯度下降算法提供了局部速率刻画。

摘要:

我们研究带平衡正则化项$\frac18\|L^\top L-R^\top R\|_F^2$的过参数化非对称矩阵感知问题的因式分解梯度下降(FGD)算法。总体目标的精确不定提升将FGD的动力学分解为正、负和零谱子空间。前两个分量具有线性恢复项,而零分量仅以二次方式收缩。对于独立同分布的各向同性亚高斯感知矩阵和具有尺度代理$\sigma$的独立中心亚高斯噪声,在足够小的初始化误差、线性测量数量$n\gtrsim\kappa^2(d_1+d_2)\max\{k^2,\sigma^2/\sigma_r^2\}$和步长$\eta\lesssim\sigma_1^{-1}$下,我们证明了局部界\\( \\|L_tR_t^\top-X^\star\\|_2 \lesssim\max\left\{\frac{1}{\eta t+1/\sigma_r}, \kappa\sigma\sqrt{\frac{d_1+d_2}{n}}\right\} \\)。此处$\sigma_1$和$\sigma_r$分别为目标$X^\star\in \mathbb{R}^{d_1\times d_2}$的最大和最小非零奇异值,且$\kappa=\sigma_1/\sigma_r$。此外,构造了一条总体轨迹以获得紧乘积误差$\Theta(1/t)$。这些结果为带平衡正则化的过参数化非对称矩阵感知问题的普通梯度下降算法提供了局部速率刻画。

英文摘要:

We study factorized gradient descent (FGD) for overparameterized asymmetric matrix sensing with the balancing regularizer $\frac18\|L^\top L-R^\top R\|_F^2$. An exact indefinite lifting of the population objective decomposes the dynamics of FGD into positive, negative, and null spectral subspaces. The first two components have linear restoring terms, whereas the null component contracts only quadratically. For independent isotropic sub-Gaussian sensing matrices and independent centered sub-Gaussian noise with scale proxy $σ$, we prove the local bound \[ \|L_tR_t^\top-X^\star\|_2 \lesssim\max\left\{\frac{1}{ηt+1/σ_r}, κσ\sqrt{\frac{d_1+d_2}{n}}\right\}, \] under a sufficiently small initialization error, the number of linear measurements $n\gtrsimκ^2(d_1+d_2)\max\{k^2,σ^2/σ_r^2\}$, and step size $η\lesssimσ_1^{-1}$. Here $σ_1$ and $σ_r$ are the largest and smallest nonzero singular values of the target $X^\star\in \mathbb{R}^{d_1\times d_2}$ respectively, and $κ=σ_1/σ_r$. Furthermore, a population trajectory was constructed to have tight product error $Θ(1/t)$. These results provide a local rate characterization for ordinary gradient descent for overparameterized asymmetric matrix sensing with balancing regularizer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑