arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.00233cs.LG
收藏

平滑铰链损失学习

Learning with Smooth Hinge Losses

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);School of Mathematical Sciences, University of Chinese Academy of Sciences(中国科学院大学数学科学学院)

JunRu Luo, Hong Qiao, Bo Zhang

摘要:

由于支持向量机中铰链损失的非光滑性,现代优化算法很难获得更快的收敛速度。本文引入了两个光滑铰链损失$ψ_G(α;σ)$和$ψ_M(α;σ)$,它们是无穷可微的,并且当$σ$趋于$0$时,它们在$α$中一致收敛于铰链损失.通过用这两个光滑的Hinge损失代替Hinge损失,我们分别得到两个光滑的支持向量机。用信赖域牛顿法(TRON)求解支持向量机,得到了两个二次收敛的算法。在文本分类任务中的实验表明,所提出的支持向量机是有效的,在现实世界中的应用。我们还引入了一个通用的光滑凸损失函数来统一机器学习中几种常用的凸损失函数。一般框架提供了光滑的近似函数的非光滑凸损失函数,它可以用来获得光滑的模型,可以解决更快的收敛优化算法。

英文摘要:

Due to the non-smoothness of the Hinge loss in SVM, it is difficult to obtain a faster convergence rate with modern optimization algorithms. In this paper, we introduce two smooth Hinge losses $ψ_G(α;σ)$ and $ψ_M(α;σ)$ which are infinitely differentiable and converge to the Hinge loss uniformly in $α$ as $σ$ tends to $0$. By replacing the Hinge loss with these two smooth Hinge losses, we obtain two smooth support vector machines(SSVMs), respectively. Solving the SSVMs with the Trust Region Newton method (TRON) leads to two quadratically convergent algorithms. Experiments in text classification tasks show that the proposed SSVMs are effective in real-world applications. We also introduce a general smooth convex loss function to unify several commonly-used convex loss functions in machine learning. The general framework provides smooth approximation functions to non-smooth convex loss functions, which can be used to obtain smooth models that can be solved with faster convergent optimization algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑