arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2208.08083cs.CV
收藏

三个臭皮匠顶个诸葛亮:鲁棒学习遇上多分支模型

Two Heads are Better than One: Robust Learning Meets Multi-branch Models

Department of Computer Science, The University of Hong Kong(计算机科学系,香港大学);Department of Electronic Engineering, Shanghai Jiao Tong University(电子工程系,上海交通大学);Department of Electrical and Electronic Engineering, The University of Hong Kong(电气电子工程系,香港大学);Department of Computer Science, City University of Hong Kong(计算机科学系,城市大学);School of Computing, National University of Singapore(计算学院,新加坡国立大学)

Zongyuan Zhang, Qingwen Bu, Tianyang Duan, Zheng Lin, Yuhao Qing, Zihan Fang, Heming Cui, Dong Huang

AI总结:

针对深度神经网络的对抗鲁棒性问题,本文提出分支正交对抗训练方法BORT,通过多分支结构加分支正交损失实现解空间正交,仅用原始数据集训练就取得了SOTA的对抗鲁棒性能。

摘要:

深度神经网络(DNNs)易受对抗样本攻击,这类包含人类难以察觉扰动的输入会误导DNNs输出错误结果。对抗训练作为可靠有效的防御方法,可显著降低神经网络的脆弱性,已成为鲁棒学习的事实标准。近期大量研究遵循以数据为中心的思路,比如探索如何生成更优质的对抗样本,或利用生成模型制作额外训练数据,而本文将视线转回模型本身,从深度特征分布的角度重新审视对抗鲁棒性,作为现有研究的有力补充。本文提出分支正交对抗训练(Branch Orthogonality adveRsarial Training, BORT),仅使用原始数据集开展对抗训练即可达到当前最优性能。为落实融合多个正交解空间的设计思路,本文采用结构简单的多分支神经网络,提出对应的分支正交损失函数,使多分支模型的各个解空间保持正交。本文分别在CIFAR-10、CIFAR-100和SVHN数据集上,针对扰动大小ε=8/255的ℓ∞范数约束扰动评估所提方法。大量实验表明,无需任何训练技巧,本文方法性能超越所有当前最优方法;与所有不使用额外数据训练的方法相比,本文模型在CIFAR-10和CIFAR-100上分别达到67.3%和41.5%的鲁棒准确率,较当前最优方法分别提升7.23%和9.07%。

英文摘要:

Deep neural networks (DNNs) are vulnerable to adversarial examples, in which DNNs are misled to false outputs due to inputs containing imperceptible perturbations. Adversarial training, a reliable and effective method of defense, may significantly reduce the vulnerability of neural networks and becomes the de facto standard for robust learning. While many recent works practice the data-centric philosophy, such as how to generate better adversarial examples or use generative models to produce additional training data, we look back to the models themselves and revisit the adversarial robustness from the perspective of deep feature distribution as an insightful complementarity. In this paper, we propose \textit{Branch Orthogonality adveRsarial Training} (BORT) to obtain state-of-the-art performance with solely the original dataset for adversarial training. To practice our design idea of integrating multiple orthogonal solution spaces, we leverage a simple multi-branch neural network and propose a corresponding loss function, branch-orthogonal loss, to make each solution space of the multi-branch model orthogonal. We evaluate our approach on CIFAR-10, CIFAR-100 and SVHN against $\ell_{\infty}$ norm-bounded perturbations of size $ε= 8/255$, respectively. Exhaustive experiments are conducted to show that our method goes beyond all state-of-the-art methods without any tricks. Compared to all methods that do not use additional data for training, our models achieve 67.3\% and 41.5\% robust accuracy on CIFAR-10 and CIFAR-100 (improving upon the state-of-the-art by +7.23\% and +9.07\%).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑