评估评估者：对对抗鲁棒性测试的信任
Evaluating the Evaluators: Trust in Adversarial Robustness Tests
AI总结：
本文提出AttackBench基准框架，在标准化可复现条件下基于新的最优性指标评估并排名基于梯度的对抗攻击，以解决现有鲁棒性测试评估不一致、不可靠并导致虚假安全感的问题。
摘要：
尽管在设计用于鲁棒性验证的强大对抗性逃避攻击方面取得了显著进展，但这些方法的评估往往仍然不一致且不可靠。许多评估依赖于不匹配的模型、未经核验的实现以及不均衡的计算预算，这可能导致有偏的结果和虚假的安全感。因此，建立在这种有缺陷的测试协议之上的鲁棒性声明可能具有误导性，并带来虚假的安全感。作为提高评估可靠性的一个具体步骤，我们提出了AttackBench，这是一个基准框架，旨在标准化和可复现的条件下评估基于梯度的攻击的有效性。AttackBench作为一个评估工具，基于一种新的最优性指标对现有攻击实现进行排名，使研究人员和从业者能够识别出最可靠和最有效的攻击，以用于后续的鲁棒性评估。该框架强制执行一致的测试条件并支持持续更新，使其成为鲁棒性验证的可靠基础。
英文摘要：
Despite significant progress in designing powerful adversarial evasion attacks for robustness verification, the evaluation of these methods often remains inconsistent and unreliable. Many assessments rely on mismatched models, unverified implementations, and uneven computational budgets, which can lead to biased results and a false sense of security. Consequently, robustness claims built on such flawed testing protocols may be misleading and give a false sense of security. As a concrete step toward improving evaluation reliability, we present AttackBench, a benchmark framework developed to assess the effectiveness of gradient-based attacks under standardized and reproducible conditions. AttackBench serves as an evaluation tool that ranks existing attack implementations based on a novel optimality metric, which enables researchers and practitioners to identify the most reliable and effective attack for use in subsequent robustness evaluations. The framework enforces consistent testing conditions and enables continuous updates, making it a reliable foundation for robustness verification.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
