arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.07089math.OC
收藏

基于交叉验证的线性SVM分类最优特征选择

Cross-validated feature selection using least-squares support vector machines for binary classification

Graduate School of Science and Technology, University of Tsukuba(筑波大学科学技术研究院);Institute of Systems and Information Engineering, University of Tsukuba(筑波大学系统信息工程学院);Gurobi Japan Co., Ltd.(Gurobi日本有限公司);Institute of Engineering, Tokyo University of Agriculture and Technology(东京农工大学工学院)

Masaharu Mori, Shunnosuke Ikeda, Ryuta Tamura, Yuichi Takano, Ryuhei Miyashiro

AI总结:

本文基于交叉验证准则扩展了最佳特征子集选择框架,将特征选择问题转化为双层混合整数优化问题,并通过LS-SVM将其简化为单层问题,实验表明该方法在分类和特征选择上表现优异。

摘要:

本文针对支持向量机(SVM)的特征子集选择问题,基于交叉验证准则进行研究。与AIC和BIC等统计准则不同,交叉验证仅需假设样本独立同分布(i.i.d.)。本文将特征选择问题转化为双层混合整数优化问题,并引入LS-SVM将其简化为单层问题,利用标准优化软件求解。通过模拟实验与正则化方法、递归特征消除和统计准则基于的MIO进行比较,结果表明所提框架在分类准确性和特征选择准确性上均表现良好。

英文摘要:

Support vector machines (SVMs) are widely used for binary classification. However, irrelevant or redundant features can degrade classification accuracy and model interpretability, making feature selection important for SVMs. Most SVM feature selection methods rely on sequential search or regularization and do not directly identify the best feature subset. Mixed-integer optimization (MIO) is a direct approach to best subset selection. Cross-validation (CV) is broadly applicable to feature selection because it evaluates predictive performance on validation data without model assumptions. Although Purcaru et al. (2026) proposed an MIO formulation for CV-based linear-SVM feature selection (SVM-CV), this method suffers from high computational cost. We develop computationally efficient CV-based MIO formulations for feature selection in linear SVM classification by using least squares support vector machines (LS-SVMs). We formulate the problem as a bilevel MIO problem, where the lower-level problem trains LS-SVM classifiers and the upper-level problem selects features based on validation sets. We then reformulate it as a single-level MIO problem by exploiting the optimality conditions of LS-SVM. We propose two formulations: a hinge-loss-based validation model and a least-squares-based validation model. We compare our methods with SVM-CV and other SVM-based and logistic-regression-based methods through simulation experiments on synthetic data. The hinge-loss-based validation model attained high feature selection accuracy under the high-noise and moderate-noise conditions, particularly with more candidate features, and the least-squares-based validation model performed well in feature selection under the low-noise condition at the lower correlation level. Furthermore, our two models achieved substantially faster computation than the existing SVM-CV model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑