arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1607.06996stat.MLcs.LG
收藏

同时进行特征和样本缩减的稀疏支持向量机

Scaling Up Sparse Support Vector Machines by Simultaneous Feature and Sample Reduction

Zhejiang University(浙江大学);Tencent AI Lab(腾讯人工智能实验室);University of Michigan(密歇根大学);University of Science and Technology of China(中国科学技术大学)

Weizhong Zhang, Bin Hong, Wei Liu, Jieping Ye, Deng Cai, Xiaofei He, Jie Wang

摘要:

稀疏支持向量机(SVM)是一种流行的分类技术,它可以同时学习一小组最可解释的特征并识别支持向量。它在许多实际应用中取得了巨大的成功。然而,对于涉及大量样本和超高维特征的大规模问题,求解稀疏支持向量机仍然具有挑战性。注意到稀疏支持向量机在特征空间和样本空间都具有稀疏性,我们提出了一种新的方法,该方法基于对稀疏支持向量机的原始和对偶最优解的精确估计,同时识别不活动的特征和样本,保证与输出无关。因此,我们可以从训练阶段中删除已识别的非活动样本和特征,从而在不牺牲准确性的情况下大幅节省计算成本。此外,我们表明,我们的方法可以扩展到多类稀疏支持向量机。据我们所知,所提出的方法是\emph{first} \emph{static}特征和样本减少方法的稀疏支持向量机和多类稀疏支持向量机。在合成数据集和真实数据集上的实验表明,我们的方法明显优于最先进的方法,并且通过我们的方法获得的加速比可以是数量级的。

英文摘要:

Sparse support vector machine (SVM) is a popular classification technique that can simultaneously learn a small set of the most interpretable features and identify the support vectors. It has achieved great successes in many real-world applications. However, for large-scale problems involving a huge number of samples and ultra-high dimensional features, solving sparse SVMs remains challenging. By noting that sparse SVMs induce sparsities in both feature and sample spaces, we propose a novel approach, which is based on accurate estimations of the primal and dual optima of sparse SVMs, to simultaneously identify the inactive features and samples that are guaranteed to be irrelevant to the outputs. Thus, we can remove the identified inactive samples and features from the training phase, leading to substantial savings in the computational cost without sacrificing the accuracy. Moreover, we show that our method can be extended to multi-class sparse support vector machines. To the best of our knowledge, the proposed method is the \emph{first} \emph{static} feature and sample reduction method for sparse SVMs and multi-class sparse SVMs. Experiments on both synthetic and real data sets demonstrate that our approach significantly outperforms state-of-the-art methods and the speedup gained by our approach can be orders of magnitude.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑