arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2002.01810cs.LGcs.CRstat.ML
收藏

理解深度神经网络的决策边界:一项实证研究

Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study

neurocat GmbH(神经猫有限责任公司)

David Mickisch, Felix Assion, Florens Greßner, Wiebke Günther, Mariele Motta

摘要:

尽管在许多图像分类任务上实现了卓越的性能,但最先进的机器学习(ML)分类器仍然容易受到小输入扰动的影响。特别是,对抗性示例的存在引发了人们对在安全和安全关键环境中部署ML模型的担忧,如自动驾驶和疾病检测。在过去的几年里,已经发布了许多防御方法,目的是提高对抗性和腐败鲁棒性。然而,拟议的措施只取得了非常有限的成功。这种有限的进展部分是由于缺乏对深度神经网络的决策边界和决策区域的理解。因此,我们研究了数据点到决策边界的最小距离,以及这个距离在深度神经网络的训练过程中如何演变。通过在MNIST、FASHION-MNIST和CIFAR-10上进行实验,我们观察到决策边界在训练过程中更接近自然图像。这种现象甚至在训练的后期保持不变,其中分类器已经获得了低的训练和测试出错率。另一方面,对抗性训练似乎有可能防止决策边界的这种不期望的收敛。

英文摘要:

Despite achieving remarkable performance on many image classification tasks, state-of-the-art machine learning (ML) classifiers remain vulnerable to small input perturbations. Especially, the existence of adversarial examples raises concerns about the deployment of ML models in safety- and security-critical environments, like autonomous driving and disease detection. Over the last few years, numerous defense methods have been published with the goal of improving adversarial as well as corruption robustness. However, the proposed measures succeeded only to a very limited extent. This limited progress is partly due to the lack of understanding of the decision boundary and decision regions of deep neural networks. Therefore, we study the minimum distance of data points to the decision boundary and how this margin evolves over the training of a deep neural network. By conducting experiments on MNIST, FASHION-MNIST, and CIFAR-10, we observe that the decision boundary moves closer to natural images over training. This phenomenon even remains intact in the late epochs of training, where the classifier already obtains low training and test error rates. On the other hand, adversarial training appears to have the potential to prevent this undesired convergence of the decision boundary.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑