正则化数据拟合的更清晰边界
Sharper Bounds for Regularized Data Fitting
摘要：
我们研究了线性回归、低秩近似和典型相关分析的正则化变量的矩阵绘制方法。我们的主要重点是草图技术，保持目标函数值的正规化问题，这是一个领域，仍然在很大程度上未被探索。我们研究正则化在一个相当广泛的设置，并在特定的背景下的流行和广泛使用的技术的脊正则化;对于后者，适用于这些问题中的每一个，我们显示算法的资源界限，其中{ \em统计维度}出现在地方，在以前的界限的排名会出现。统计维数总是小于秩，并且随着正则化量的增加而减小。特别地，对于岭型低秩逼近问题$\min_{Y,X} \lVert YX - A \rVert_F^2 + λ\lVert Y\rVert_F^2 + λ\lVert X \rVert_F^2$，其中$Y\in\mathbb{R}^{n\times k}$和$X\in\mathbb{R}^{k\times d}$，我们给出了一个需要\[ O(\mathtt{nnz}(A)) + \tilde{O}((n+d)\varepsilon^{-1}k \min\{k, \varepsilon^{-1}\mathtt{sd}_λ(Y^*)\})+ \mathtt{poly}(\mathtt{sd}_λ(Y^*) \varepsilon^{-1}) \]时间的逼近算法，其中$s_λ(Y^*)\le k$是$Y^*$的统计维数，$Y^*$是最优$Y$，$\varepsilon$是误差参数，$\mathtt{nnz}(A)$是$A$的非零项个数.我们还在更一般的环境中研究正则化。例如，我们获得了基于草图的算法的低秩近似问题$\min_{X,Y} \lVert YX - A \rVert_F^2 + f(Y,X)$，其中$f(\cdot,\cdot)$是一个正则化函数满足一些非常一般的条件（主要是，正交变换下的不变性）。
英文摘要：
We study matrix sketching methods for regularized variants of linear regression, low rank approximation, and canonical correlation analysis. Our main focus is on sketching techniques which preserve the objective function value for regularized problems, which is an area that has remained largely unexplored. We study regularization both in a fairly broad setting, and in the specific context of the popular and widely used technique of ridge regularization; for the latter, as applied to each of these problems, we show algorithmic resource bounds in which the {\em statistical dimension} appears in places where in previous bounds the rank would appear. The statistical dimension is always smaller than the rank, and decreases as the amount of regularization increases. In particular, for the ridge low-rank approximation problem $\min_{Y,X} \lVert YX - A \rVert_F^2 + λ\lVert Y\rVert_F^2 + λ\lVert X \rVert_F^2$, where $Y\in\mathbb{R}^{n\times k}$ and $X\in\mathbb{R}^{k\times d}$, we give an approximation algorithm needing \[ O(\mathtt{nnz}(A)) + \tilde{O}((n+d)\varepsilon^{-1}k \min\{k, \varepsilon^{-1}\mathtt{sd}_λ(Y^*)\})+ \mathtt{poly}(\mathtt{sd}_λ(Y^*) \varepsilon^{-1}) \] time, where $s_λ(Y^*)\le k$ is the statistical dimension of $Y^*$, $Y^*$ is an optimal $Y$, $\varepsilon$ is an error parameter, and $\mathtt{nnz}(A)$ is the number of nonzero entries of $A$.This is faster than prior work, even when $λ=0$. We also study regularization in a much more general setting. For example, we obtain sketching-based algorithms for the low-rank approximation problem $\min_{X,Y} \lVert YX - A \rVert_F^2 + f(Y,X)$ where $f(\cdot,\cdot)$ is a regularizing function satisfying some very general conditions (chiefly, invariance under orthogonal transformations).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
