岭回归与可证确定性岭杠杆分数抽样
Ridge Regression and Provable Deterministic Ridge Leverage Score Sampling
摘要：
岭杠杆分数提供了低秩近似和正则化之间的平衡，并且在随机线性代数和机器学习中无处不在。确定性算法在中等规模的大数据体系中也很受关注，因为确定性算法通过没有失败概率并且总是返回相同的结果来为从业者提供可解释性。我们提供了可证明的保证确定性列抽样使用岭杠杆得分。由我们的算法返回的矩阵草图是原始矩阵的列子集，从而产生额外的可解释性。与随机化算法类似，确定性算法提供（1 + ε）错误列子集选择、（1 + ε）错误投影-成本保持以及加性-乘性谱界。我们还表明，在幂律衰减的岭杠杆分数的假设下，这种确定性算法是可证明的随机算法一样准确。最后，岭回归经常用于正则化不适定的线性最小二乘问题。虽然岭回归提供了回归系数的收缩，但许多系数仍然很小但不为零。使用我们的算法返回的矩阵草图和特定的正则化参数执行岭回归，迫使系数为零，并且在统计风险上具有可证明的（1 + ε）界。因此，它是弹性网络正则化的一个有趣的替代方案。
英文摘要：
Ridge leverage scores provide a balance between low-rank approximation and regularization, and are ubiquitous in randomized linear algebra and machine learning. Deterministic algorithms are also of interest in the moderately big data regime, because deterministic algorithms provide interpretability to the practitioner by having no failure probability and always returning the same results. We provide provable guarantees for deterministic column sampling using ridge leverage scores. The matrix sketch returned by our algorithm is a column subset of the original matrix, yielding additional interpretability. Like the randomized counterparts, the deterministic algorithm provides (1 + ε) error column subset selection, (1 + ε) error projection-cost preservation, and an additive-multiplicative spectral bound. We also show that under the assumption of power-law decay of ridge leverage scores, this deterministic algorithm is provably as accurate as randomized algorithms. Lastly, ridge regression is frequently used to regularize ill-posed linear least-squares problems. While ridge regression provides shrinkage for the regression coefficients, many of the coefficients remain small but non-zero. Performing ridge regression with the matrix sketch returned by our algorithm and a particular regularization parameter forces coefficients to zero and has a provable (1 + ε) bound on the statistical risk. As such, it is an interesting alternative to elastic net regularization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
