利用异构教师蒸馏对抗鲁棒性
Distilling Adversarial Robustness Using Heterogeneous Teachers
AI总结：
本文提出 DARHT 框架，通过利用多个低对抗样本可迁移性的异构教师进行知识蒸馏，在 CIFAR-10、CIFAR-100 和 Tiny ImageNet 上实现了领先的干净与鲁棒准确率。
摘要：
在错误分类会造成重大代价的领域（例如自动驾驶汽车或医学影像）中部署神经网络分类器之前，实现抵御对抗攻击的弹性是必要的。近期工作已经证明，可以通过知识蒸馏将鲁棒性从经过对抗训练的教师迁移到学生模型。然而，当前方法使用单个对抗教师和普通教师进行蒸馏，并考虑同构架构（即残差网络），这些架构容易对来自相似对抗子空间的样本产生错误分类。在这项工作中，我们通过使用异构教师蒸馏对抗鲁棒性（DARHT）开发了一个对抗攻击防御框架。在 DARHT 中，学生模型在学生-教师特征图中显式表示教师 logits，并利用多个表现出低对抗样本可迁移性（即在不相似的对抗样本上表现出高性能）的教师。在白盒和黑盒场景下的分类任务实验表明，与 CIFAR-10、CIFAR-100 和 Tiny ImageNet 数据集上竞争的对抗训练和蒸馏方法相比，DARHT 实现了最先进的干净准确率和鲁棒准确率。与同构和异构教师集的比较表明，利用具有低对抗样本可迁移性的教师可以提高学生模型的鲁棒性。
英文摘要：
Achieving resiliency against adversarial attacks is necessary prior to deploying neural network classifiers in domains where misclassification incurs substantial costs, e.g., self-driving cars or medical imaging. Recent work has demonstrated that robustness can be transferred from an adversarially trained teacher to a student model using knowledge distillation. However, current methods perform distillation using a single adversarial and vanilla teacher and consider homogeneous architectures (i.e., residual networks) that are susceptible to misclassify examples from similar adversarial subspaces. In this work, we develop a defense framework against adversarial attacks by distilling adversarial robustness using heterogeneous teachers (DARHT). In DARHT, the student model explicitly represents teacher logits in a student-teacher feature map and leverages multiple teachers that exhibit low adversarial example transferability (i.e., exhibit high performance on dissimilar adversarial examples). Experiments on classification tasks in both white-box and black-box scenarios demonstrate that DARHT achieves state-of-the-art clean and robust accuracies when compared to competing adversarial training and distillation methods in the CIFAR-10, CIFAR-100, and Tiny ImageNet datasets. Comparisons with homogeneous and heterogeneous teacher sets suggest that leveraging teachers with low adversarial example transferability increases student model robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
