arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.00447cs.CVcs.CRcs.LG
收藏

可验证鲁棒神经网络的评价与训练

Towards Evaluating and Training Verifiably Robust Neural Networks

SenseTime-CUHK Joint Lab(商汤-香港中文大学联合实验室);The Chinese University of Hong Kong(香港中文大学);Centre of Perceptual and Interactive Intelligence(感知与交互智能研究中心)

Zhaoyang Lyu, Minghao Guo, Tong Wu, Guodong Xu, Kehuan Zhang, Dahua Lin

摘要:

最近的工作表明,区间界限传播(IBP)可以用于训练可验证的鲁棒神经网络。研究人员在这些IBP训练的网络上观察到一个有趣的现象:CROWN,一种基于紧线性松弛的边界方法,通常会在这些网络上给出非常宽松的边界。我们还观察到大多数神经元在IBP训练过程中死亡,这可能会损害网络的表示能力。本文研究了IBP和CROWN之间的关系,证明了在选择合适的边界线时,CROWN总是比IBP紧。我们进一步提出了一个宽松版本的皇冠,线性边界传播(LBP),可用于验证大型网络,以获得较低的验证错误比IBP。我们还设计了一个新的激活函数,参数化斜坡函数(ParamRamp),它比ReLU具有更多的神经元状态的多样性。我们在MNIST,CIFAR-10和Tiny-ImageNet上进行了广泛的实验,并使用ParamRamp激活,实现了最先进的验证鲁棒性。代码和附录可在https://github.com/ZhaoyangLyu/VerifiablyRobustNN上获得。

英文摘要:

Recent works have shown that interval bound propagation (IBP) can be used to train verifiably robust neural networks. Reseachers observe an intriguing phenomenon on these IBP trained networks: CROWN, a bounding method based on tight linear relaxation, often gives very loose bounds on these networks. We also observe that most neurons become dead during the IBP training process, which could hurt the representation capability of the network. In this paper, we study the relationship between IBP and CROWN, and prove that CROWN is always tighter than IBP when choosing appropriate bounding lines. We further propose a relaxed version of CROWN, linear bound propagation (LBP), that can be used to verify large networks to obtain lower verified errors than IBP. We also design a new activation function, parameterized ramp function (ParamRamp), which has more diversity of neuron status than ReLU. We conduct extensive experiments on MNIST, CIFAR-10 and Tiny-ImageNet with ParamRamp activation and achieve state-of-the-art verified robustness. Code and the appendix are available at https://github.com/ZhaoyangLyu/VerifiablyRobustNN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑