具有算子范数误差的降秩回归
Reduced-Rank Regression with Operator Norm Error
摘要:
一个常见的数据分析任务是降秩回归问题:$$\min_{\textrm{rank-}k \ X} \|AX-B\|,$$,其中$A \in \mathbb{R}^{n \times c}$和$B \in \mathbb{R}^{n \times d}$是给定的大矩阵,$\|\cdot\|$是某个范数。这里,未知矩阵$X \in \mathbb{R}^{c \times d}$被约束为秩为$k$,因为当$c$和$d$较大时,它导致解的参数显著减少。在Frobenius范数误差的情况下,有一个标准的封闭形式的解决方案,这个问题和一个快速算法找到一个$(1+\varepsilon)$近似的解决方案。然而,对于算子范数误差的重要情况,没有已知的封闭形式的解决方案,并且已知的最快算法需要奇异值分解时间。我们给这个问题的第一个随机算法运行时间$$(nnz{(A)} + nnz{(B)} + c^2) \cdot k/\varepsilon^{1.5} + (n+d)k^2/ε+ c^ω,$$的多对数因子,涉及条件数,矩阵维数,并依赖于$1/\varepsilon$。这里,$nnz{(M)}$表示矩阵$M$的非零条目的数目,并且$ω$是矩阵乘法的指数。由于(1)谱低秩近似($A = B$)和(2)线性系统求解($n = c$和$d = 1$)都是特殊情况,如果线性代数没有重大突破,我们的时间不能提高超过一个$1/\varepsilon$因子(直到多对数因子)。有趣的是,用于低秩近似的已知技术,例如交替最小化或草图和解决,可证明对这个问题失败。相反,我们的算法使用的解决方案的存在性表征,连同Krylov方法,低次多项式近似,和草图为基础的预处理。
英文摘要:
A common data analysis task is the reduced-rank regression problem: $$\min_{\textrm{rank-}k \ X} \|AX-B\|,$$ where $A \in \mathbb{R}^{n \times c}$ and $B \in \mathbb{R}^{n \times d}$ are given large matrices and $\|\cdot\|$ is some norm. Here the unknown matrix $X \in \mathbb{R}^{c \times d}$ is constrained to be of rank $k$ as it results in a significant parameter reduction of the solution when $c$ and $d$ are large. In the case of Frobenius norm error, there is a standard closed form solution to this problem and a fast algorithm to find a $(1+\varepsilon)$-approximate solution. However, for the important case of operator norm error, no closed form solution is known and the fastest known algorithms take singular value decomposition time. We give the first randomized algorithms for this problem running in time $$(nnz{(A)} + nnz{(B)} + c^2) \cdot k/\varepsilon^{1.5} + (n+d)k^2/ε+ c^ω,$$ up to a polylogarithmic factor involving condition numbers, matrix dimensions, and dependence on $1/\varepsilon$. Here $nnz{(M)}$ denotes the number of non-zero entries of a matrix $M$, and $ω$ is the exponent of matrix multiplication. As both (1) spectral low rank approximation ($A = B$) and (2) linear system solving ($n = c$ and $d = 1$) are special cases, our time cannot be improved by more than a $1/\varepsilon$ factor (up to polylogarithmic factors) without a major breakthrough in linear algebra. Interestingly, known techniques for low rank approximation, such as alternating minimization or sketch-and-solve, provably fail for this problem. Instead, our algorithm uses an existential characterization of a solution, together with Krylov methods, low degree polynomial approximation, and sketching-based preconditioning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。