arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.09441cs.LG
收藏

对抗性微调对压缩神经网络影响的实证研究

An Empirical Study of the Influence of Adversarial Fine-Tuning on Compressed Neural Networks

Department of Computer Science, University of Copenhagen, Denmark(丹麦哥本哈根大学计算机科学系)

Hallgrimur Thorsteinsson, Valdemar J Henriksen, Daniel I R Cruz, Raghavendra Selvan, Tong Chen

AI总结:

通过实验研究压缩模型的对抗性微调,发现其能显著提升鲁棒性,并在计算效率与鲁棒性之间取得平衡。

摘要:

随着深度学习模型日益融入日常生活,通过使其抵御对抗性攻击来确保安全性变得至关重要。研究发现,通过引入微小、有针对性的扰动来干扰输入数据,深度学习模型容易受到对抗性攻击。对抗性训练作为一种缓解策略,可以产生更鲁棒的模型。然而,这种对抗鲁棒性伴随着训练过程中设计对抗性攻击所需的额外计算成本。因此,对抗鲁棒性和计算效率这两个目标似乎相互冲突。在这项工作中,我们探讨了神经网络压缩对对抗鲁棒性的影响。我们特别研究了微调对压缩模型的影响,并展示了标准微调与对抗性微调之间的权衡。我们的结果表明,对压缩模型进行对抗性微调可以大幅提升其鲁棒性性能。我们在多个基准数据集上进行了实验,表明压缩模型的对抗性微调可以达到与对抗性训练模型相当的鲁棒性性能,同时提高计算效率。源代码可在此处获取:https://github.com/saintslab/Adver-Fine。

英文摘要:

As deep learning (DL) models are increasingly being integrated into our everyday lives, ensuring their safety by making them robust against adversarial attacks has become increasingly critical. DL models have been found to be susceptible to adversarial attacks by introducing small, targeted perturbations to disrupt the input data. Adversarial training has been presented as a mitigation strategy that can result in more robust models. This adversarial robustness comes with additional computational costs required to design adversarial attacks during training. The two objectives -- adversarial robustness and computational efficiency -- then appear to be in conflict with each other. In this work, we explore the effects of neural network compression on adversarial robustness. We specifically explore the effects of fine-tuning on compressed models, and present the trade-off between standard fine-tuning and adversarial fine-tuning. Our results show that adversarial fine-tuning of compressed models can yield large improvements to their robustness performance. We present experiments on several benchmark datasets showing that adversarial fine-tuning of compressed models can achieve robustness performance comparable to adversarially trained models, while also improving computational efficiency. Source code is available here: https://github.com/saintslab/Adver-Fine.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑