arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1809.09350cs.LGstat.ML
收藏

全内隐在线学习

Fully Implicit Online Learning

Tsinghua University(清华大学);Tencent AI Lab(腾讯AI实验室)

Chaobing Song, Ji Liu, Han Liu, Yong Jiang, Tong Zhang

摘要:

正则化在线学习广泛应用于机器学习应用中。在在线学习中,执行精确最小化($i.e.,$隐式更新)已知有利于解的数值稳定性和结构。本文研究了一类正则化在线算法,我们称之为\emph{fully implicit online learning}(FIOL)。我们证明了对于任意Bregman发散,FIOL在一般凸情形下具有$O(\sqrt{T})$遗憾,在强凸情形下具有$O(\log T)$遗憾,并且该遗憾避免了线性化的逼近误差,具有一步改进效果.然后,我们提出了有效的算法来解决FIOL的子问题。我们表明,即使子问题的解决方案没有封闭的形式,它可以解决的线性化在线算法的复杂性。实验验证了所提出的方法。

英文摘要:

Regularized online learning is widely used in machine learning applications. In online learning, performing exact minimization ($i.e.,$ implicit update) is known to be beneficial to the numerical stability and structure of solution. In this paper we study a class of regularized online algorithms without linearizing the loss function or the regularizer, which we call \emph{fully implicit online learning} (FIOL). We show that for arbitrary Bregman divergence, FIOL has the $O(\sqrt{T})$ regret for general convex setting and $O(\log T)$ regret for strongly convex setting, and the regret has an one-step improvement effect because it avoids the approximation error of linearization. Then we propose efficient algorithms to solve the subproblem of FIOL. We show that even if the solution of the subproblem has no closed form, it can be solved with complexity comparable to the linearized online algoritms. Experiments validate the proposed approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑