arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2003.13370stat.MLcs.LG
收藏

抗大扰动的深度学习模型

Towards Deep Learning Models Resistant to Large Perturbations

Sharif University of Technology(谢里夫理工大学)

Amirreza Shaeiri, Rozhin Nobahari, Mohammad Hossein Rohban

摘要:

对抗鲁棒性已被证明是机器学习算法的必要属性。这个问题的一个关键且经常被忽视的方面是试图使对抗性噪声幅度尽可能大,以增强模型鲁棒性的好处。我们证明了一个被称为“对抗训练”的成熟算法在给定一个大的但合理的扰动幅度的情况下,无法训练一个深度神经网络。在本文中,我们提出了一种简单而有效的网络权重初始化方法,使在更高噪声水平上学习成为可能。接下来,我们在MNIST($ε$到$\approx 0.40$)和CIFAR10($ε$到$\approx 32/255$)数据集上严格评估这个想法,假设$\ell_{\infty}$攻击模型。此外,为了建立学习可行的$ε$的限制,我们研究了最佳的鲁棒分类器,假设完全访问联合数据和标签分布。然后,我们提供了一个简单的多维伯努利分布的对抗精度的一些理论结果,这产生了一些见解的可行的扰动范围的MNIST数据集。

英文摘要:

Adversarial robustness has proven to be a required property of machine learning algorithms. A key and often overlooked aspect of this problem is to try to make the adversarial noise magnitude as large as possible to enhance the benefits of the model robustness. We show that the well-established algorithm called "adversarial training" fails to train a deep neural network given a large, but reasonable, perturbation magnitude. In this paper, we propose a simple yet effective initialization of the network weights that makes learning on higher levels of noise possible. We next evaluate this idea rigorously on MNIST ($ε$ up to $\approx 0.40$) and CIFAR10 ($ε$ up to $\approx 32/255$) datasets assuming the $\ell_{\infty}$ attack model. Additionally, in order to establish the limits of $ε$ in which the learning is feasible, we study the optimal robust classifier assuming full access to the joint data and label distribution. Then, we provide some theoretical results on the adversarial accuracy for a simple multi-dimensional Bernoulli distribution, which yields some insights on the range of feasible perturbations for the MNIST dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑