arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.01736cs.LGcs.AI
收藏

几何感知的实例重加权对抗训练

Geometry-aware Instance-reweighted Adversarial Training

RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心);National University of Singapore(新加坡国立大学);Hong Kong Baptist University(香港浸会大学);The University of Tokyo(东京大学)

Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, Mohan Kankanhalli

摘要:

在对抗性机器学习中,人们普遍认为鲁棒性和准确性是互相伤害的。最近的研究挑战了这一信念,我们可以保持鲁棒性并提高准确性。然而,另一个方向,我们是否可以在提高鲁棒性的同时保持准确性,在概念上和实际上更有趣,因为对于任何模型,鲁棒准确性都应该低于标准准确性。在本文中,我们表明这一方向也是有希望的。首先,我们发现即使是过度参数化的深度网络也可能仍然没有足够的模型容量,因为对抗性训练具有压倒性的平滑效果。其次,考虑到有限的模型容量,我们认为对抗性数据应该具有不等的重要性:从几何上讲,更接近/更远离类边界的自然数据点更不健壮/更健壮,相应的对抗性数据点应该分配更大/更小的权重。最后,为了实现这个想法,我们提出了几何感知的实例重新加权对抗训练,其中权重基于攻击自然数据点的难度。实验表明,我们的建议提高了标准对抗训练的鲁棒性;结合两个方向,我们提高了标准对抗训练的鲁棒性和准确性。

英文摘要:

In adversarial machine learning, there was a common belief that robustness and accuracy hurt each other. The belief was challenged by recent studies where we can maintain the robustness and improve the accuracy. However, the other direction, whether we can keep the accuracy while improving the robustness, is conceptually and practically more interesting, since robust accuracy should be lower than standard accuracy for any model. In this paper, we show this direction is also promising. Firstly, we find even over-parameterized deep networks may still have insufficient model capacity, because adversarial training has an overwhelming smoothing effect. Secondly, given limited model capacity, we argue adversarial data should have unequal importance: geometrically speaking, a natural data point closer to/farther from the class boundary is less/more robust, and the corresponding adversarial data point should be assigned with larger/smaller weight. Finally, to implement the idea, we propose geometry-aware instance-reweighted adversarial training, where the weights are based on how difficult it is to attack a natural data point. Experiments show that our proposal boosts the robustness of standard adversarial training; combining two directions, we improve both robustness and accuracy of standard adversarial training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑