arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1705.05067cs.LG
收藏

鲁棒频繁方向及其在在线学习中的应用

Robust Frequent Directions with Application in Online Learning

Shanghai Jiao Tong University(上海交通大学);Peking University(北京大学);Nanjing University(南京大学);Hong Kong University of Science and Technology(香港科技大学)

Luo Luo, Cheng Chen, Zhihua Zhang, Wu-Jun Li, Tong Zhang

摘要:

频繁方向(FD)技术是一种确定性的在线草图绘制方法,在机器学习中有许多应用。传统的FD是一个启发式的过程,经常输出秩亏矩阵。为了克服秩不足的问题,我们提出了一种新的草图策略称为鲁棒频繁方向(RFD)通过引入一个正则化项。RFD可以从优化问题中导出。它自适应地联合更新草图矩阵和正则化项。RFD在不增加计算量的情况下减小了FD的近似误差。我们还将RFD应用到在线学习中,提出了一种有效的无超参数在线牛顿算法。我们推导出了基于RFD的在线牛顿算法的遗憾界,保证了算法的鲁棒性。实验结果表明,该方法优于现有的二阶在线学习算法。

英文摘要:

The frequent directions (FD) technique is a deterministic approach for online sketching that has many applications in machine learning. The conventional FD is a heuristic procedure that often outputs rank deficient matrices. To overcome the rank deficiency problem, we propose a new sketching strategy called robust frequent directions (RFD) by introducing a regularization term. RFD can be derived from an optimization problem. It updates the sketch matrix and the regularization term adaptively and jointly. RFD reduces the approximation error of FD without increasing the computational cost. We also apply RFD to online learning and propose an effective hyperparameter-free online Newton algorithm. We derive a regret bound for our online Newton algorithm based on RFD, which guarantees the robustness of the algorithm. The experimental studies demonstrate that the proposed method outperforms state-of-the-art second order online learning algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑