基于随机自集成的鲁棒神经网络
Towards Robust Neural Networks via Random Self-ensemble
摘要:
最近的研究揭示了深度神经网络的脆弱性:一个人类无法察觉的小的对抗性扰动很容易使训练有素的深度神经网络错误分类。这使得在安全关键应用中应用神经网络变得不安全。本文结合{ \bf随机性}和{ \bf集成}这两个重要概念,提出了一种新的防御算法--随机自集成(RSE)。为了保护目标模型,RSE在神经网络中加入随机噪声层以防止基于梯度的强攻击,并在随机噪声上集成预测以稳定性能。我们证明了该算法等价于集成无限多个噪声模型$f_ε$而不需要任何额外的内存开销,并且基于噪声随机梯度下降的训练过程可以确保集成模型具有良好的预测能力.我们的算法显着优于以前的防御技术对真实数据集。以CIFAR-10为例,在VGG网络上(未攻击时预测准确率为92%),在一定的失真容限内,在强C &W攻击下,未保护模型的预测准确率下降到10%以下,以往最好的防御技术具有$48\%$的预测准确率,而我们的方法在同样的攻击水平下仍具有$86\%$的预测准确率。最后,我们的方法是简单的,易于集成到任何神经网络。
英文摘要:
Recent studies have revealed the vulnerability of deep neural networks: A small adversarial perturbation that is imperceptible to human can easily make a well-trained deep neural network misclassify. This makes it unsafe to apply neural networks in security-critical applications. In this paper, we propose a new defense algorithm called Random Self-Ensemble (RSE) by combining two important concepts: {\bf randomness} and {\bf ensemble}. To protect a targeted model, RSE adds random noise layers to the neural network to prevent the strong gradient-based attacks, and ensembles the prediction over random noises to stabilize the performance. We show that our algorithm is equivalent to ensemble an infinite number of noisy models $f_ε$ without any additional memory overhead, and the proposed training procedure based on noisy stochastic gradient descent can ensure the ensemble model has a good predictive capability. Our algorithm significantly outperforms previous defense techniques on real data sets. For instance, on CIFAR-10 with VGG network (which has 92\% accuracy without any attack), under the strong C\&W attack within a certain distortion tolerance, the accuracy of unprotected model drops to less than 10\%, the best previous defense technique has $48\%$ accuracy, while our method still has $86\%$ prediction accuracy under the same level of attack. Finally, our method is simple and easy to integrate into any neural network.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。