DATABench: 从对抗角度评估深度学习中的数据集审计
DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective
AI总结：
DATABench从对抗角度评估深度学习中的数据集审计，提出新的分类法和攻击策略，揭示现有审计方法在对抗环境下的不足。
摘要：
深度学习在各个领域广泛应用，其关键取决于训练数据集的质量和组成。然而，数据集使用情况的普遍不透明性引发了严重的隐私和版权问题。旨在确定特定数据集是否用于训练给定可疑模型的数据集审计技术，为解决这些透明性缺口提供了有前景的解决方案。尽管先前的工作开发了各种审计方法，但它们对专门对抗性攻击的鲁棒性仍鲜有探索。为弥补这一差距，本文发起了一项全面研究，从对抗角度评估数据集审计。我们首先介绍了一种新的分类法，根据方法对内部特征（IF）（数据固有）与外部特征（EF）（为审计人工引入）的依赖性对现有方法进行分类。随后，我们提出了两种主要攻击类型：逃避攻击，旨在掩盖数据集的使用；以及伪造攻击，旨在错误地指控未使用的数据集。基于现有方法和攻击目标的理解，我们进一步提出了系统化的攻击策略：解耦、移除和检测用于逃避；对抗示例方法用于伪造。这些公式和策略导致了我们的新基准DATABench，包含17种逃避攻击、5种伪造攻击和9种代表性审计方法。使用DATABench的广泛评估表明，评估的审计方法在对抗设置下均不够鲁棒或独特。这些发现突显了开发更安全、更可靠的数据集审计方法的紧迫性，以抵御复杂的对抗性操纵。代码可在https://github.com/shaoshuo-ss/DATABench获得。
英文摘要：
The widespread application of Deep Learning across diverse domains hinges critically on the quality and composition of training datasets. However, the common lack of disclosure regarding their usage raises significant privacy and copyright concerns. Dataset auditing techniques, which aim to determine if a specific dataset was used to train a given suspicious model, provide promising solutions to addressing these transparency gaps. While prior work has developed various auditing methods, their resilience against dedicated adversarial attacks remains largely unexplored. To bridge the gap, this paper initiates a comprehensive study evaluating dataset auditing from an adversarial perspective. We start with introducing a novel taxonomy, classifying existing methods based on their reliance on internal features (IF) (inherent to the data) versus external features (EF) (artificially introduced for auditing). Subsequently, we formulate two primary attack types: evasion attacks, designed to conceal the use of a dataset, and forgery attacks, intending to falsely implicate an unused dataset. Building on the understanding of existing methods and attack objectives, we further propose systematic attack strategies: decoupling, removal, and detection for evasion; adversarial example-based methods for forgery. These formulations and strategies lead to our new benchmark, DATABench, comprising 17 evasion attacks, 5 forgery attacks, and 9 representative auditing methods. Extensive evaluations using DATABench reveal that none of the evaluated auditing methods are sufficiently robust or distinctive under adversarial settings. These findings underscore the urgent need for developing a more secure and reliable dataset auditing method capable of withstanding sophisticated adversarial manipulation. Code is available in https://github.com/shaoshuo-ss/DATABench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
