arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1303.1849cs.LGcs.DScs.NAmath.NA
收藏

重新审视 Nyström 方法以改进大规模机器学习

Revisiting the Nystrom Method for Improved Large-Scale Machine Learning

California Institute of Technology(加州理工学院);Stanford University(斯坦福大学)

Alex Gittens, Michael W. Mahoney

AI总结:

本文重新审视了对称半正定矩阵低秩近似的随机算法,通过经验评估和理论分析比较了采样与投影方法的性能,并提出了更优的误差界。

摘要:

我们重新考虑了数据分析和机器学习应用中出现的对称半正定(SPSD)矩阵(如拉普拉斯矩阵和核矩阵)低秩近似的随机算法。我们的主要结果包括对一系列多样化的SPSD矩阵上采样和投影方法的性能质量和运行时间的经验评估。我们的结果突出了采样与投影方法的互补性;它们描述了常见数据预处理步骤对这些算法性能的影响;并指出了基于杠杆得分的均匀采样与非均匀采样方法之间的重要差异。此外,我们的经验结果表明,现有理论非常薄弱,甚至无法为实践提供定性指导。因此,我们用一套随机采样和随机投影方法的最坏情况理论界限来补充我们的经验结果。这些界限在质量上优于现有界限——例如,谱范数和Frobenius范数误差的改进加性误差界,以及迹范数误差的相对误差界——并指出了使这些算法在更大规模机器学习应用中更有用的未来方向。

英文摘要:

We reconsider randomized algorithms for the low-rank approximation of symmetric positive semi-definite (SPSD) matrices such as Laplacian and kernel matrices that arise in data analysis and machine learning applications. Our main results consist of an empirical evaluation of the performance quality and running time of sampling and projection methods on a diverse suite of SPSD matrices. Our results highlight complementary aspects of sampling versus projection methods; they characterize the effects of common data preprocessing steps on the performance of these algorithms; and they point to important differences between uniform sampling and nonuniform sampling methods based on leverage scores. In addition, our empirical results illustrate that existing theory is so weak that it does not provide even a qualitative guide to practice. Thus, we complement our empirical results with a suite of worst-case theoretical bounds for both random sampling and random projection methods. These bounds are qualitatively superior to existing bounds---e.g. improved additive-error bounds for spectral and Frobenius norm error and relative-error bounds for trace norm error---and they point to future directions to make these algorithms useful in even larger-scale machine learning applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑