arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1605.01656stat.MLcs.ITcs.LGcs.NAmath.ITmath.NAmath.OC
收藏

一个硬约束

A Tight Bound of Hard Thresholding

Rutgers University(罗格斯大学);Baidu Research(百度研究院)

Jie Shen, Ping Li

摘要:

本文研究的是一种硬阈值算子,它将矢量中除$k$最大绝对值元素外的所有元素都设置为零。我们建立了一个{ \em紧}约束来定量表征给定信号的阈值解决方案的偏差。我们的理论结果是普遍的,因为它适用于所有参数的选择,并且基本的分析仅依赖于数学优化中的基本参数。我们讨论了两个领域的影响:压缩感知。由于至关重要的估计,我们之间的连接桥梁的限制等距属性(RIP)和稀疏参数的大量的硬阈值为基础的算法,这使得RIP条件的改善,特别是当真正的稀疏性是未知的。这表明,在本质上,更多种类的传感矩阵或更少的测量是可接受的数据采集过程。机器学习。在大规模机器学习方面,一个重要但具有挑战性的问题是以有效的方式学习精确的稀疏模型。在形成鲜明对比,以前的工作,试图促进稀疏性的$\ell_1$ -松弛,我们提出了一种新的随机算法,在每次迭代中执行硬阈值,从而确保这种简约的解决方案。在较弱的假设下,我们证明了一些流行的统计模型的{ \em全局线性收敛},即使问题是非凸的.

英文摘要:

This paper is concerned with the hard thresholding operator which sets all but the $k$ largest absolute elements of a vector to zero. We establish a {\em tight} bound to quantitatively characterize the deviation of the thresholded solution from a given signal. Our theoretical result is universal in the sense that it holds for all choices of parameters, and the underlying analysis depends only on fundamental arguments in mathematical optimization. We discuss the implications for two domains: Compressed Sensing. On account of the crucial estimate, we bridge the connection between the restricted isometry property (RIP) and the sparsity parameter for a vast volume of hard thresholding based algorithms, which renders an improvement on the RIP condition especially when the true sparsity is unknown. This suggests that in essence, many more kinds of sensing matrices or fewer measurements are admissible for the data acquisition procedure. Machine Learning. In terms of large-scale machine learning, a significant yet challenging problem is learning accurate sparse models in an efficient manner. In stark contrast to prior work that attempted the $\ell_1$-relaxation for promoting sparsity, we present a novel stochastic algorithm which performs hard thresholding in each iteration, hence ensuring such parsimonious solutions. Equipped with the developed bound, we prove the {\em global linear convergence} for a number of prevalent statistical models under mild assumptions, even though the problem turns out to be non-convex.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑