AttackBench：评估用于对抗样本的基于梯度的攻击
AttackBench: Evaluating Gradient-based Attacks for Adversarial Examples
AI总结：
本文提出 AttackBench 基准框架，通过最优性指标和统一查询预算，对百余种基于梯度的对抗攻击进行公平的有效性与效率评估。
摘要：
对抗样本通常通过基于梯度的攻击进行优化。尽管新攻击不断被提出，但每种攻击都是在不同的实验设置、超参数配置以及对目标模型的前向和反向调用次数下被证明优于其前代方法。这导致了过于乐观甚至带有偏差的评估，可能不公平地偏向某一种特定攻击。在本研究中，我们旨在通过提出 AttackBench 来克服这些局限，它是首个能够在不同攻击之间进行公平比较的评估框架。为此，我们首先对基于梯度的攻击进行分类，识别其主要组成部分和差异。随后介绍我们的框架，该框架评估这些攻击的有效性与效率。我们通过以下方式衡量这些特性：（i）定义一个最优性指标，量化攻击与最优解的接近程度；（ii）限制对模型的前向和反向查询次数，使所有攻击都在给定的最大查询预算内进行比较。我们的大量实验分析在 CIFAR-10 和 ImageNet 模型上比较了 100 多种攻击实现以及总计 800 多种不同配置，结果表明只有极少数攻击优于所有竞争方法。在该分析中，我们揭示了若干实现问题，这些问题导致许多攻击无法找到更优解，甚至根本无法运行。我们将 AttackBench 作为公开可用的基准发布，旨在持续更新，纳入并评估用于优化对抗样本的新型基于梯度的攻击。
英文摘要：
Adversarial examples are typically optimized with gradient-based attacks. While novel attacks are continuously proposed, each is shown to outperform its predecessors using different experimental setups, hyperparameter settings, and number of forward and backward calls to the target models. This provides overly-optimistic and even biased evaluations that may unfairly favor one particular attack over the others. In this work, we aim to overcome these limitations by proposing AttackBench, i.e., the first evaluation framework that enables a fair comparison among different attacks. To this end, we first propose a categorization of gradient-based attacks, identifying their main components and differences. We then introduce our framework, which evaluates their effectiveness and efficiency. We measure these characteristics by (i) defining an optimality metric that quantifies how close an attack is to the optimal solution, and (ii) limiting the number of forward and backward queries to the model, such that all attacks are compared within a given maximum query budget. Our extensive experimental analysis compares more than $100$ attack implementations with a total of over $800$ different configurations against CIFAR-10 and ImageNet models, highlighting that only very few attacks outperform all the competing approaches. Within this analysis, we shed light on several implementation issues that prevent many attacks from finding better solutions or running at all. We release AttackBench as a publicly-available benchmark, aiming to continuously update it to include and evaluate novel gradient-based attacks for optimizing adversarial examples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
