arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.01473cs.LGcs.AI
收藏

符号约束支持向量机的学习

Learning Sign-Constrained Support Vector Machines

Gunma University(群马大学)

Kenya Tajima, Takahiko Henmi, Kohei Tsuchida, Esmeraldo Ronnie R. Zara, Tsuyoshi Kato

摘要:

领域知识对于提高学习机的泛化性能是非常有用的。符号约束是将领域知识与学习机结合起来的一种方便的表示方法。本文在线性支持向量机的学习过程中,考虑对权系数的符号进行约束,提出了两种在符号约束下最小化经验风险的优化算法。其中一种算法是基于投影梯度法的,投影梯度法的每次迭代都需要$O(nd)$的计算代价,并且保证了目标误差的次线性收敛。第二个算法是基于Frank-Wolfe方法,也收敛于次线性,并具有明确的终止准则。我们表明,每次迭代的Frank-Wolfe也需要$O(nd)$成本。通过分析目标函数的曲率,得到了保证解具有$ε$精度的最小迭代次数的显式表达式。最后,我们的经验表明,符号约束是一个很有前途的技术时,相似的训练样本组成的特征向量。

英文摘要:

Domain knowledge is useful to improve the generalization performance of learning machines. Sign constraints are a handy representation to combine domain knowledge with learning machine. In this paper, we consider constraining the signs of the weight coefficients in learning the linear support vector machine, and develop two optimization algorithms for minimizing the empirical risk under the sign constraints. One of the two algorithms is based on the projected gradient method, in which each iteration of the projected gradient method takes $O(nd)$ computational cost and the sublinear convergence of the objective error is guaranteed. The second algorithm is based on the Frank-Wolfe method that also converges sublinearly and possesses a clear termination criterion. We show that each iteration of the Frank-Wolfe also requires $O(nd)$ cost. Furthermore, we derive the explicit expression for the minimal iteration number to ensure an $ε$-accurate solution by analyzing the curvature of the objective function. Finally, we empirically demonstrate that the sign constraints are a promising technique when similarities to the training examples compose the feature vector.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑