arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1905.12418cs.LGcs.CRstat.ML
收藏

鲁棒训练的期望紧界

Expected Tight Bounds for Robust Training

KAUST(阿卜杜拉国王科技大学);University of Oxford(牛津大学)

Salman Alsubaihi, Adel Bibi, Modar Alfadly, Abdullah Hamdi, Bernard Ghanem

摘要:

训练对范数有界对抗攻击具有鲁棒性的深度神经网络仍然是一个难以捉摸的问题。虽然基于精确和不精确验证的方法通常过于昂贵,无法训练大型网络,但已经证明,有界输入区间可以通过深度网络从一层传播到另一层。这种区间界限传播方法(IBP)不仅提高了鲁棒性和认证准确性,而且是第一个在大型/深度网络上使用的方法。然而,由于IBP边界的非常松散的性质,所需的训练过程是复杂和复杂的。在本文中,我们仔细研究了以Affine-ReLU-Affine形式组成的层块的边界。为此,我们提出了预期的严格界限(真正的界限的期望),被称为ETB,这是可证明的严格比IBP界限的期望。然后,我们通过逐块传播将这一结果扩展到更深的网络,并表明与IBP相比,我们可以实现数量级更紧的边界。此外,使用简单的标准训练过程,我们可以在MNIST和CIFAR 10上实现令人印象深刻的鲁棒性-准确性权衡。

英文摘要:

Training Deep Neural Networks that are robust to norm bounded adversarial attacks remains an elusive problem. While exact and inexact verification-based methods are generally too expensive to train large networks, it was demonstrated that bounded input intervals can be inexpensively propagated from a layer to another through deep networks. This interval bound propagation approach (IBP) not only has improved both robustness and certified accuracy but was the first to be employed on large/deep networks. However, due to the very loose nature of the IBP bounds, the required training procedure is complex and involved. In this paper, we closely examine the bounds of a block of layers composed in the form of Affine-ReLU-Affine. To this end, we propose expected tight bounds (true bounds in expectation), referred to as ETB, which are provably tighter than IBP bounds in expectation. We then extend this result to deeper networks through blockwise propagation and show that we can achieve orders of magnitudes tighter bounds compared to IBP. Furthermore, using a simple standard training procedure, we can achieve impressive robustness-accuracy trade-off on both MNIST and CIFAR10.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑