arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1808.01688cs.CV
收藏

鲁棒性是准确性的代价吗?--18种深度图像分类模型鲁棒性综合研究

Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models

IBM Research(IBM研究);University of California, Davis(加利福尼亚大学戴维斯分校);Massachusetts Institute of Technology(麻省理工学院);JD AI Research(京东AI研究院)

Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, Yupeng Gao

摘要:

预测精度一直是比较不同图像分类模型性能的长期和唯一标准,包括ImageNet竞赛。然而,最近的研究强调了训练有素的深度神经网络对对抗性示例缺乏鲁棒性。对自然图像的视觉上不可感知的扰动可以很容易地被制作并误导图像分类器进行错误分类。为了揭示鲁棒性和准确性之间的权衡,本文使用多个鲁棒性指标对18个ImageNet模型进行了全面的基准测试,包括306对模型之间对抗性示例的失真,成功率和可转移性。我们广泛的实验结果揭示了几个新的见解:(1)线性缩放定律-经验$\ell_2$和$\ell_\infty$失真度量与分类错误的对数线性缩放;(2)模型架构是比模型大小更关键的鲁棒性因素,并且所公开的准确性-鲁棒性Pareto前沿可以用作ImageNet模型设计者的评估标准;(3)对于类似的网络架构,增加网络深度略微提高了$\ell_\infty$失真的鲁棒性;(4)存在表现出高对抗可转移性的模型(在VGG家族中),而从一个模型制作的大多数对抗示例只能在同一家族中转移。实验代码可在\url{https://github.com/huanzhang12/Adversarial_Survey}上公开获得。

英文摘要:

The prediction accuracy has been the long-lasting and sole standard for comparing the performance of different image classification models, including the ImageNet competition. However, recent studies have highlighted the lack of robustness in well-trained deep neural networks to adversarial examples. Visually imperceptible perturbations to natural images can easily be crafted and mislead the image classifiers towards misclassification. To demystify the trade-offs between robustness and accuracy, in this paper we thoroughly benchmark 18 ImageNet models using multiple robustness metrics, including the distortion, success rate and transferability of adversarial examples between 306 pairs of models. Our extensive experimental results reveal several new insights: (1) linear scaling law - the empirical $\ell_2$ and $\ell_\infty$ distortion metrics scale linearly with the logarithm of classification error; (2) model architecture is a more critical factor to robustness than model size, and the disclosed accuracy-robustness Pareto frontier can be used as an evaluation criterion for ImageNet model designers; (3) for a similar network architecture, increasing network depth slightly improves robustness in $\ell_\infty$ distortion; (4) there exist models (in VGG family) that exhibit high adversarial transferability, while most adversarial examples crafted from one model can only be transferred within the same family. Experiment code is publicly available at \url{https://github.com/huanzhang12/Adversarial_Survey}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑