arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1009.4791cs.LG
收藏

实例加权支持向量机的多参数求解路径算法

Multi-parametric Solution-path Algorithm for Instance-weighted Support Vector Machines

Nagoya Institute of Technology(名古屋工业大学);Tokyo Institute of Technology(东京工业大学)

Masayuki Karasuyama, Naoyuki Harada, Masashi Sugiyama, Ichiro Takeuchi

摘要:

支持向量机的实例加权变体最近引起了相当大的关注,因为它们在各种机器学习任务中是有用的,例如非平稳数据分析、异方差数据建模、迁移学习、学习排序和转换。在这些情况下,一个重要的挑战是克服计算瓶颈-实例权重经常动态或自适应地变化,因此加权SVM解决方案必须重复计算。在本文中,我们开发了一种算法,可以有效地和准确地更新加权SVM解决方案的任意变化的实例权重。从技术上讲,这一贡献可以被视为传统的解决方案路径算法的一个单一的正则化参数的多个实例权重参数的扩展。然而,这种扩展引起了一个重要的问题,即断点(在该解决方案的路径转向)必须在高维空间中确定。为了方便这一点,我们引入了实例权重的参数化表示。我们还提供了一个几何解释权空间中使用的概念的关键区域:一个多面体,其中目前的仿射解决方案仍然是最佳的。然后,我们发现断点的解决方案的路径和边界的多面体的交叉点。通过大量的实验,在各种实际应用中,我们证明了该算法的实用性。

英文摘要:

An instance-weighted variant of the support vector machine (SVM) has attracted considerable attention recently since they are useful in various machine learning tasks such as non-stationary data analysis, heteroscedastic data modeling, transfer learning, learning to rank, and transduction. An important challenge in these scenarios is to overcome the computational bottleneck---instance weights often change dynamically or adaptively, and thus the weighted SVM solutions must be repeatedly computed. In this paper, we develop an algorithm that can efficiently and exactly update the weighted SVM solutions for arbitrary change of instance weights. Technically, this contribution can be regarded as an extension of the conventional solution-path algorithm for a single regularization parameter to multiple instance-weight parameters. However, this extension gives rise to a significant problem that breakpoints (at which the solution path turns) have to be identified in high-dimensional space. To facilitate this, we introduce a parametric representation of instance weights. We also provide a geometric interpretation in weight space using a notion of critical region: a polyhedron in which the current affine solution remains to be optimal. Then we find breakpoints at intersections of the solution path and boundaries of polyhedrons. Through extensive experiments on various practical applications, we demonstrate the usefulness of the proposed algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑