The support vector machine (SVM) is one of the most successful learning methods for solving classification problems. Despite its popularity, SVM has a serious drawback, that is sensitivity to outliers in training samples. The penalty on misclassification is defined by a convex loss called the hinge loss, and the unboundedness of the convex loss causes the sensitivity to outliers. To deal with outliers, robust variants of SVM have been proposed, such as the robust outlier detection algorithm and an SVM with a bounded loss called the ramp loss. In this paper, we propose a robust variant of SVM and investigate its robustness in terms of the breakdown point. The breakdown point is a robustness measure that is the largest amount of contamination such that the estimated classifier still gives information about the non-contaminated data. The main contribution of this paper is to show an exact evaluation of the breakdown point for the robust SVM. For learning parameters such as the regularization parameter in our algorithm, we derive a simple formula that guarantees the robustness of the classifier. When the learning parameters are determined with a grid search using cross validation, our formula works to reduce the number of candidate search points. The robustness of the proposed method is confirmed in numerical experiments. We show that the statistical properties of the robust SVM are well explained by a theoretical analysis of the breakdown point.
Outlier-robust neural network training: variation regularization meets trimmed loss to prevent functional breakdown
Institute of Statistical Mathematics（统计数学研究所）；The Graduate University for Advanced Studies, SOKENDAI（高等研究大学、SOKENDAI）；RIKEN（理化学研究所）；Joint Support-Center for Data Science Research（数据科学联合支持中心）
击穿点的概念是由Hampel [Ph.D.论文（1968），加州大学伯克利分校; Ann. Math. Statistist. 42（1971）1887-1896]并由Huber等人进一步发展[稳健统计（1981）。Wiley，New York] and Donoho and Huber [In A Festschrift for Erich L. Lehmann（1983）157-184. Wadsworth，Belmont，CA].事实证明，它在解决地点、规模和回归问题方面最为成功。试图将这一概念扩大到其他情况的努力没有得到普遍接受。在本文中，我们认为，这是连接到一个事实，即在位置，规模和回归问题的平移和仿射群产生的定义统计泛函的等方差。比较方面的崩溃点似乎只有有用的限制时，等变泛函，甚至在这里之间的连接崩溃和等变是一个脆弱的。
ICSI（国际计算机科学研究所）；UC Berkeley（加州大学伯克利分校）；National University of Singapore（新加坡国立大学）；Technion（以色列理工学院）
我们提出了一个分布式鲁棒统计学习{ \em大污染数据}的框架。分布式鲁棒学习（DRL）框架可以将传统鲁棒学习方法的计算时间减少几个数量级。我们分析了DRL的鲁棒性，表明DRL不仅保留了基础鲁棒学习方法的鲁棒性，而且还容忍计算节点（节点故障）结果的恒定部分上的污染。更确切地说，即使在计算节点上存在最具对抗性的离群值分布的情况下，DRL仍然实现至少$ λ^*/2 $的分解点，其中$ λ^* $是对应的集中式算法的分解点。这与朴素的除法和平均实现方式形成鲜明对比，朴素的除法和平均实现方式在使用$ k $计算节点时可以将崩溃点减少$ k $的因子。然后，我们专门的DRL框架的两个具体情况下：分布式鲁棒主成分分析和分布式鲁棒回归。我们证明了DRL的效率和鲁棒性的优势，通过全面的模拟和预测图像标签上的大规模图像集。
