arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.13541cs.CR
收藏

DWBench: 图像数据集版权审计中水印的综合评估

DWBench: Holistic Evaluation of Watermark for Dataset Copyright Auditing

Zhejiang University(浙江大学);Xi’an Jiaotong University(西安交通大学);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学)

Xiao Ren, Xinyi Yu, Linkang Du, Min Chen, Yuanchao Shu, Zhou Su, Yunjun Gao, Zhikun Zhang

AI总结:

DWBench通过统一基准和开源工具系统评估图像数据集水印技术,揭示了不同任务中方法的权衡与局限性。

摘要:

随着深度学习中大规模数据集需求的激增,数据所有者面临未经授权使用带来的风险,因此需要有效的版权保护措施。尽管数据集水印技术在审计和验证使用方面具有潜力,但现有方法受限于不一致的评估,阻碍了公平比较和实际应用的评估。为解决这一差距,我们提出了一种两层分类法,按实现方式(基于模型 vs. 基于模型的注入;基于模型行为 vs. 基于模型消息的验证)对方法进行分类,提供了一个结构化的跨任务分析框架。然后,我们开发了DWBench,一个统一的基准和开源工具包,用于系统评估图像数据集水印技术在分类和生成任务中的表现。使用DWBench,我们按照标准化条件、基于扰动的鲁棒性测试、多水印共存和多用户干扰评估了25种代表性方法。除了报告四种常用指标的结果外,我们还展示了两种新指标的结果:样本重要性用于细粒度水印可区分性,以及用于数据集层面审计的验证成功率,这使基准测试更加准确和可重复。关键发现揭示了固有的权衡:没有单一方法在所有场景中占主导地位;分类和生成任务需要专门的方法;现有技术在低水印率和现实多用户设置中表现出不稳定,出现较高的误报或性能下降。我们希望DWBench能促进水印可靠性和实用性的发展,从而在面对广泛的人工智能驱动的数据滥用时加强版权保护。

英文摘要:

The surging demand for large-scale datasets in deep learning has heightened the need for effective copyright protection, given the risks of unauthorized use to data owners. Although the dataset watermark technique holds promise for auditing and verifying usage, existing methods are hindered by inconsistent evaluations, which impede fair comparisons and assessments of real-world viability. To address this gap, we organize existing methods according to two key dimensions, implementation and verification, to support a consistent analysis and evaluation pipeline across tasks. Based on this framework, we develop DWBench, a unified benchmark and open-source toolkit for systematically evaluating image dataset watermark techniques in classification and generation tasks. Using DWBench, we assess 25 representative methods under standardized conditions, perturbation-based robustness tests, multi-watermark coexistence, and multi-user interference. To enable accurate and reproducible benchmarking, we use TPR@5%FPR for unified sample-level comparison and introduce the verification success rate (VSR) for dataset-level auditing. Key findings reveal that standard single-watermark evaluations tend to overestimate practical auditability. Methods that verify reliably in isolation often suffer from performance degradation at low watermarked-sample ratios, while yielding ambiguous ownership evidence in complex multi-user and multi-watermark settings. We hope that DWBench can facilitate advances in watermark reliability and practicality, thus strengthening copyright safeguards in the face of widespread AI-driven data exploitation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑