基于自适应超范数正则化的多类SVM的变量选择
Variable selection for the multicategory SVM via adaptive sup-norm regularization
摘要:
支持向量机(SVM)是机器学习中一种流行的分类方法,在实际应用中取得了巨大的成功。然而,标准的SVM不能自动选择变量,因此其解决方案通常利用所有的输入变量没有歧视。这使得识别重要的预测变量变得困难,而这通常是数据分析的主要目标之一。在本文中,我们提出了两种新类型的正则化的多类别SVM(MSVM)的背景下,同时分类和变量选择。MSVM通常需要估计多个判别函数,并应用argmax规则进行预测。对于每个单独的变量,我们建议其重要性的特征与不同的功能,其系数向量的supnorm,然后最小化的MSVM铰链损失函数受到惩罚的supnorm的总和。为了进一步改善超范数惩罚,我们提出了自适应正则化,它允许根据不同的变量的相对重要性施加不同的权重。这两种类型的正则化在构建分类器的过程中自动进行变量选择,并导致具有增强的可解释性和提高的准确性的稀疏多分类器,特别是对于高维低样本量数据。超范数惩罚的一个很大的优点是它很容易通过标准线性规划实现。几个模拟的例子和一个真实的基因数据分析表明,在各种数据设置的自适应超范数惩罚的出色表现。
英文摘要:
The Support Vector Machine (SVM) is a popular classification paradigm in machine learning and has achieved great success in real applications. However, the standard SVM can not select variables automatically and therefore its solution typically utilizes all the input variables without discrimination. This makes it difficult to identify important predictor variables, which is often one of the primary goals in data analysis. In this paper, we propose two novel types of regularization in the context of the multicategory SVM (MSVM) for simultaneous classification and variable selection. The MSVM generally requires estimation of multiple discriminating functions and applies the argmax rule for prediction. For each individual variable, we propose to characterize its importance by the supnorm of its coefficient vector associated with different functions, and then minimize the MSVM hinge loss function subject to a penalty on the sum of supnorms. To further improve the supnorm penalty, we propose the adaptive regularization, which allows different weights imposed on different variables according to their relative importance. Both types of regularization automate variable selection in the process of building classifiers, and lead to sparse multi-classifiers with enhanced interpretability and improved accuracy, especially for high dimensional low sample size data. One big advantage of the supnorm penalty is its easy implementation via standard linear programming. Several simulated examples and one real gene data analysis demonstrate the outstanding performance of the adaptive supnorm penalty in various data settings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。