arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1610.03045cs.LGmath.OCstat.ML
收藏

速写遇到对偶随机投影:一种可证明的高维大数据恢复算法

Sketching Meets Random Projection in the Dual: A Provable Recovery Algorithm for Big and High-dimensional Data

University of Chicago(芝加哥大学);University of Southern California(南加州大学);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)

Jialei Wang, Jason D. Lee, Mehrdad Mahdavi, Mladen Kolar, Nathan Srebro

摘要:

草图技术已经变得流行,用于通过减少大量数据集的样本大小或维度来扩展机器学习算法,同时仍然保持大数据的统计能力。本文从优化的角度研究了草图绘制问题:首先证明了迭代Hessian草图是一个带预处理的优化过程,并通过共轭方向的搜索提出了加速迭代Hessian草图;然后,我们建立了Hessian草图和对偶随机投影之间的原始-对偶联系,并将预条件共轭梯度法应用于对偶问题,这导致加速迭代对偶随机投影方法。最后,为了解决来自大样本量和高维的挑战,我们提出了原始-对偶草图,它迭代地绘制原始和对偶公式。我们表明,使用对数调用的小规模问题的求解器,原始对偶草图是能够恢复到任意精度的原始问题的最佳。所提出的算法进行了验证,通过大量的实验合成和真实的数据集,补充了我们的理论结果。

英文摘要:

Sketching techniques have become popular for scaling up machine learning algorithms by reducing the sample size or dimensionality of massive data sets, while still maintaining the statistical power of big data. In this paper, we study sketching from an optimization point of view: we first show that the iterative Hessian sketch is an optimization process with preconditioning, and develop accelerated iterative Hessian sketch via the searching the conjugate direction; we then establish primal-dual connections between the Hessian sketch and dual random projection, and apply the preconditioned conjugate gradient approach on the dual problem, which leads to the accelerated iterative dual random projection methods. Finally to tackle the challenges from both large sample size and high-dimensionality, we propose the primal-dual sketch, which iteratively sketches the primal and dual formulations. We show that using a logarithmic number of calls to solvers of small scale problem, primal-dual sketch is able to recover the optimum of the original problem up to arbitrary precision. The proposed algorithms are validated via extensive experiments on synthetic and real data sets which complements our theoretical results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑