arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1401.0086stat.ML
收藏

基数约束下一般凸光滑函数的向前向后贪婪算法

Forward-Backward Greedy Algorithms for General Convex Smooth Functions over A Cardinality Constraint

Ji Liu, Ryohei Fujimaki, Jieping Ye

摘要:

我们考虑用前向-后向贪婪算法解决一般凸光滑函数的稀疏特征选择问题。前向-后向贪婪算法(Forward-Backward greedy algorithm,FoBa-obj)是一种最先进的贪婪算法,需要求解大量的优化问题,因此它不适合大规模问题。FoBa-gdt算法在每次前向迭代时使用梯度信息进行特征选择,显著提高了FoBa-obj算法的效率。本文系统地分析了这两种前向-后向贪婪算法的理论性质。我们的主要贡献是:1)对于一般光滑凸函数,我们得到了比现有的关于FoBa-obj的分析更好的理论界; 2)我们证明了FoBa-gdt在相同的条件下达到了与FoBa-obj相同的理论性能:限制强凸性条件。我们的新界与一个特殊情况的界是一致的3)证明了当独立样本数大于$\bar{k}\log d$时,满足约束强凸性条件,其中$\bar{k}$是稀疏数,$d$是变量的维数; 4)将FoBa-gdt(具有条件随机场目标)应用于室内活动识别的传感器选择问题,实验结果表明FoBa-gdt优于其他方法(包括基于前向贪婪选择和L1正则化的方法)。

英文摘要:

We consider forward-backward greedy algorithms for solving sparse feature selection problems with general convex smooth functions. A state-of-the-art greedy method, the Forward-Backward greedy algorithm (FoBa-obj) requires to solve a large number of optimization problems, thus it is not scalable for large-size problems. The FoBa-gdt algorithm, which uses the gradient information for feature selection at each forward iteration, significantly improves the efficiency of FoBa-obj. In this paper, we systematically analyze the theoretical properties of both forward-backward greedy algorithms. Our main contributions are: 1) We derive better theoretical bounds than existing analyses regarding FoBa-obj for general smooth convex functions; 2) We show that FoBa-gdt achieves the same theoretical performance as FoBa-obj under the same condition: restricted strong convexity condition. Our new bounds are consistent with the bounds of a special case (least squares) and fills a previously existing theoretical gap for general convex smooth functions; 3) We show that the restricted strong convexity condition is satisfied if the number of independent samples is more than $\bar{k}\log d$ where $\bar{k}$ is the sparsity number and $d$ is the dimension of the variable; 4) We apply FoBa-gdt (with the conditional random field objective) to the sensor selection problem for human indoor activity recognition and our results show that FoBa-gdt outperforms other methods (including the ones based on forward greedy selection and L1-regularization).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑