学习模型中的边界厚度和鲁棒性
Boundary thickness and robustness in learning models
摘要：
机器学习模型对各种对抗性和非对抗性腐败的鲁棒性继续受到关注。在本文中，我们介绍了分类器的边界厚度的概念，我们描述了它的连接和有用的模型鲁棒性。厚的决策边界导致改进的性能，而薄的决策边界导致过拟合（例如，通过训练和测试之间的鲁棒泛化间隙来测量）和较低的鲁棒性。我们表明，较厚的边界有助于提高对抗性示例的鲁棒性（例如，提高对抗训练的鲁棒测试精度）以及所谓的分布外（OOD）变换，并且我们表明许多常用的正则化和数据增强程序可以增加边界厚度。在理论方面，我们确定在训练期间最大化边界厚度类似于所谓的混合训练。使用这些观察结果，我们表明，混合训练的噪声增强进一步增加了边界厚度，从而对抗各种形式的对抗攻击和OOD变换的脆弱性。我们还可以证明，最近几行工作中的性能改进与更厚的边界有关。
英文摘要：
Robustness of machine learning models to various adversarial and non-adversarial corruptions continues to be of interest. In this paper, we introduce the notion of the boundary thickness of a classifier, and we describe its connection with and usefulness for model robustness. Thick decision boundaries lead to improved performance, while thin decision boundaries lead to overfitting (e.g., measured by the robust generalization gap between training and testing) and lower robustness. We show that a thicker boundary helps improve robustness against adversarial examples (e.g., improving the robust test accuracy of adversarial training) as well as so-called out-of-distribution (OOD) transforms, and we show that many commonly-used regularization and data augmentation procedures can increase boundary thickness. On the theoretical side, we establish that maximizing boundary thickness during training is akin to the so-called mixup training. Using these observations, we show that noise-augmentation on mixup training further increases boundary thickness, thereby combating vulnerability to various forms of adversarial attacks and OOD transforms. We can also show that the performance improvement in several lines of recent work happens in conjunction with a thicker boundary.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
