arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.12765cs.CV
收藏

通过增强弱子网来提高鲁棒性

Improving Robustness by Enhancing Weak Subnets

Max Planck Institute for Informatics(马克斯·普朗克信息学研究所)

Yong Guo, David Stutz, Bernt Schiele

摘要:

尽管它们取得了成功,但深度网络已被证明极易受到扰动的影响,通常会导致准确性显著下降。在本文中,我们研究模型的鲁棒性扰动输入通过研究内部子网络的性能。有趣的是,我们观察到,大多数的干扰显示特别差的鲁棒性对扰动。更重要的是,这些弱的稳定性与整体缺乏鲁棒性相关。针对这一现象,我们提出了一种新的训练过程,可以识别和增强弱神经元(EWS),以提高鲁棒性。具体来说,我们开发了一种搜索算法,以找到特别弱的网络,并明确加强他们通过知识蒸馏从整个网络。我们表明,EWS大大提高了对损坏图像的鲁棒性以及对干净数据的准确性。作为流行的数据增强方法的补充,EWS与这些方法相结合时,不断提高鲁棒性。为了突出我们方法的灵活性,我们还将EWS与流行的对抗训练方法相结合,从而提高了对抗鲁棒性。

英文摘要:

Despite their success, deep networks have been shown to be highly susceptible to perturbations, often causing significant drops in accuracy. In this paper, we investigate model robustness on perturbed inputs by studying the performance of internal sub-networks (subnets). Interestingly, we observe that most subnets show particularly poor robustness against perturbations. More importantly, these weak subnets are correlated with the overall lack of robustness. Tackling this phenomenon, we propose a new training procedure that identifies and enhances weak subnets (EWS) to improve robustness. Specifically, we develop a search algorithm to find particularly weak subnets and explicitly strengthen them via knowledge distillation from the full network. We show that EWS greatly improves both robustness against corrupted images as well as accuracy on clean data. Being complementary to popular data augmentation methods, EWS consistently improves robustness when combined with these approaches. To highlight the flexibility of our approach, we combine EWS also with popular adversarial training methods resulting in improved adversarial robustness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑