基于半光滑牛顿法的核支持向量机线性化与逼近方法
A Semismooth-Newton's-Method-Based Linearization and Approximation Approach for Kernel Support Vector Machines
摘要：
支持向量机（SVM）是最流行和性能最好的分类算法之一。已经提出了各种方法来减少在使用核支持向量机基于大规模数据集进行训练和预测时的高计算和存储成本。线性化框架是一种流行的框架，它成功地在$L_1$ -损失核支持向量机和$L_1$ -损失线性支持向量机之间架起了一座桥梁。对于线性支持向量机，最近提出了一种半光滑牛顿法。它被证明是非常有竞争力的，并具有较低的计算成本。因此，一个自然的问题是，是否有可能开发一个快速的半光滑牛顿算法的核支持向量机。基于这一问题和线性化框架的思想，本文以$L_2$ -损失核SVM为研究对象，提出了一种基于半光滑牛顿法的线性化和逼近方法，其主要思想是首先建立一个等价的线性SVM，然后利用Nyström方法对核矩阵进行逼近，得到一个约简的线性SVM。最后，采用快速半光滑牛顿法求解约简线性支持向量机。对核矩阵的逼近问题进行了理论分析。所提出的方法的优点是，它保持了较低的计算成本，并保持了快速的收敛速度。大量的数值实验结果验证了所提出的方法在预测精度和速度方面的有效性。
英文摘要：
Support Vector Machines (SVMs) are among the most popular and the best performing classification algorithms. Various approaches have been proposed to reduce the high computation and memory cost when training and predicting based on large-scale datasets with kernel SVMs. A popular one is the linearization framework, which successfully builds a bridge between the $L_1$-loss kernel SVM and the $L_1$-loss linear SVM. For linear SVMs, very recently, a semismooth Newton's method is proposed. It is shown to be very competitive and have low computational cost. Consequently, a natural question is whether it is possible to develop a fast semismooth Newton's algorithm for kernel SVMs. Motivated by this question and the idea in linearization framework, in this paper, we focus on the $L_2$-loss kernel SVM and propose a semismooth Newton's method based linearization and approximation approach for it. The main idea of this approach is to first set up an equivalent linear SVM, then apply the Nyström method to approximate the kernel matrix, based on which a reduced linear SVM is obtained. Finally, the fast semismooth Newton's method is employed to solve the reduced linear SVM. We also provide some theoretical analyses on the approximation of the kernel matrix. The advantage of the proposed approach is that it maintains low computational cost and keeps a fast convergence rate. Results of extensive numerical experiments verify the efficiency of the proposed approach in terms of both predicting accuracy and speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
