arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.11103cs.LGcs.CR
收藏

对抗攻击的性能评估:离散性与解决方案

Performance Evaluation of Adversarial Attacks: Discrepancies and Solutions

University of Electronic Science and Technology of China(电子科技大学);Monash University(莫纳什大学)

Jing Wu, Mingyi Zhou, Ce Zhu, Yipeng Liu, Mehrtash Harandi, Li Li

摘要:

最近,已经开发了对抗攻击方法来挑战机器学习模型的鲁棒性。然而,主流评价标准存在局限性,甚至在不同环境下产生结果差异。通过研究各种攻击算法,包括基于梯度和基于查询的攻击,我们注意到缺乏一个统一的标准,公正的性能评估的共识。因此,我们提出了一个分段采样曲线(PSC)工具包,以有效地解决上述差异,通过在给定范围内的对手之间生成一个全面的比较。此外,方案支助费用工具包提供了平衡计算费用和评价效力的备选办法。实验结果表明,我们的PSC工具包提供了全面的攻击算法的比较,显着减少在实践中的差异。

英文摘要:

Recently, adversarial attack methods have been developed to challenge the robustness of machine learning models. However, mainstream evaluation criteria experience limitations, even yielding discrepancies among results under different settings. By examining various attack algorithms, including gradient-based and query-based attacks, we notice the lack of a consensus on a uniform standard for unbiased performance evaluation. Accordingly, we propose a Piece-wise Sampling Curving (PSC) toolkit to effectively address the aforementioned discrepancy, by generating a comprehensive comparison among adversaries in a given range. In addition, the PSC toolkit offers options for balancing the computational cost and evaluation effectiveness. Experimental results demonstrate our PSC toolkit presents comprehensive comparisons of attack algorithms, significantly reducing discrepancies in practice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑