arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.01769cs.LGcs.AImath.OC
收藏

一种具有双层优化的 $ν$ 支持向量机安全筛选规则

A Safe Screening Rule with Bi-level Optimization of $ν$ Support Vector Machine

College of Statistics and Mathematics, Yunnan University of Finance and Economics(云南财经大学统计与数学学院);College of Science, China Agricultural University(中国农业大学理学院)

Zhiji Yang, Wanyi Chen, Huan Zhang, Yitian Xu, Lei Shi, Jianhua Zhao

AI总结:

针对 $ν$-SVM 大规模训练开销问题,提出结合 KKT 条件、变分不等式和 $ν$-性质的双层优化安全筛选规则及 DCDM,并以统一框架推广到单类 SVM。

摘要:

支持向量机(SVM)在机器学习中取得了许多成功,尤其适用于小样本问题。作为传统 SVM 的著名扩展,$ν$ 支持向量机($ν$-SVM)因其出色的模型可解释性而表现出卓越性能。然而,在面向大规模问题时,它仍面临训练开销方面的挑战。为解决这一问题,我们提出了一种用于 $ν$-SVM 的双层优化安全筛选规则(SRBO-$ν$-SVM),该规则能够在训练前筛选出非活跃样本,并在不牺牲预测精度的情况下降低计算成本。我们的 SRBO-$ν$-SVM 通过结合 Karush-Kuhn-Tucker(KKT)条件、凸问题的变分不等式以及 $ν$-性质进行严格推导。此外,我们开发了一种高效的对偶坐标下降法(DCDM),以进一步提高计算速度。最后,提出了一个用于 SRBO 的统一框架,以加速许多 SVM 类型模型,并将其成功应用于单类 SVM。在 6 个人工数据集和 30 个基准数据集上的实验结果验证了我们所提方法在监督和无监督任务中的有效性与安全性。

英文摘要:

Support vector machine (SVM) has achieved many successes in machine learning, especially for a small sample problem. As a famous extension of the traditional SVM, the $ν$ support vector machine ($ν$-SVM) has shown outstanding performance due to its great model interpretability. However, it still faces challenges in training overhead for large-scale problems. To address this issue, we propose a safe screening rule with bi-level optimization for $ν$-SVM (SRBO-$ν$-SVM) which can screen out inactive samples before training and reduce the computational cost without sacrificing the prediction accuracy. Our SRBO-$ν$-SVM is strictly deduced by integrating the Karush-Kuhn-Tucker (KKT) conditions, the variational inequalities of convex problems and the $ν$-property. Furthermore, we develop an efficient dual coordinate descent method (DCDM) to further improve computational speed. Finally, a unified framework for SRBO is proposed to accelerate many SVM-type models, and it is successfully applied to one-class SVM. Experimental results on 6 artificial data sets and 30 benchmark data sets have verified the effectiveness and safety of our proposed methods in supervised and unsupervised tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑