基于塌陷生成的文本到图像扩散模型指纹识别
Fingerprinting Text-to-Image Diffusion Models via Collapsed Generation
- Fudan University（复旦大学）
- East China University of Science and Technology（华东理工大学）
- Alibaba Group（阿里巴巴集团）
- Shanghai Pudong Research Institute of Cryptology（上海浦东密码学研究所）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出基于塌陷生成的非侵入式指纹框架，可在白盒和黑盒设置下验证文本到图像扩散模型的所有权，且对微调衍生模型及混淆具有鲁棒性。
AI中文摘要：
专有文本到图像扩散模型越来越多地以托管服务和可下载检查点的形式分发，当涉及模型泄露、复制或未经授权的微调时，其知识产权（IP）保护成为日益关键的问题。在本研究中，我们提出了一种基于塌陷生成（collapsed generation）的非侵入式模型指纹框架，塌陷生成是指某些输入条件在多个随机种子下产生高度一致图像的现象。我们表明，塌陷生成是学习生成过程的固有、依赖模型的属性，因此这些易塌陷条件会暴露模型特定的行为特征，无需嵌入侵入式水印即可实现可靠的所有权验证。在准备源模型的条件后，该框架在两种访问设置下验证可疑模型：（1）白盒流水线访问，可将优化的连续嵌入注入生成过程；（2）仅API的黑盒访问，通过服务接口查询自然语言提示。在这两种情况下，所有权证据通过可疑模型在随机采样中是否重现源模型的塌陷行为来衡量。对基于UNet和Transformer的扩散模型进行的大量实验表明，塌陷生成指纹能够以低混淆度区分不同的源模型，这些指纹在微调衍生模型以及常见和自适应的模型或查询级混淆下仍可验证，且仅需适度的验证查询预算。综上，这些结果确立了塌陷生成作为非侵入式扩散模型所有权验证的可靠固有证据源。
英文摘要：
Proprietary text-to-image diffusion models are increasingly distributed as hosted services and downloadable checkpoints, making their intellectual property (IP) protection an increasingly critical concern when model leakage, copying, or unauthorized fine-tuning is disputed. In this work, we present a non-invasive model fingerprinting framework based on \emph{collapsed generation}, a phenomenon where certain input conditions produce highly consistent images across multiple stochastic seeds. We show that collapsed generation is an intrinsic, model-dependent property of the learned generation process. These collapse-prone conditions therefore expose model-specific behavioral signatures, enabling reliable ownership verification without embedding invasive watermarks. After preparing conditions on the source model, the framework verifies a suspect model under two access settings: (1) white-box pipeline access, where optimized continuous embeddings can be injected into the generation process, and (2) black-box API-only access, where natural language prompts are queried through the service interface. In both cases, ownership evidence is measured by whether the suspect model reproduces the source model's collapse behavior across stochastic samplings. Extensive experiments across UNet- and transformer-based diffusion models show that collapsed generation fingerprints can distinguish different source models with low confusion. These fingerprints remain verifiable in fine-tuned derivatives and under common and adaptive model- or query-level obfuscations, while requiring only a modest verification query budget. Together, these results establish collapsed generation as a reliable intrinsic evidence source for non-invasive diffusion model ownership verification.
