arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.02376cs.CV
收藏

文本引导的图像到图像扩散模型的来源识别

Origin Identification for Text-Guided Image-to-Image Diffusion Models

University of Technology Sydney(悉尼技术大学);Peking University(北京大学);Harvard University(哈佛大学);Zhejiang University(浙江大学)

Wenhao Wang, Yifan Sun, Zongxin Yang, Zhentao Tan, Zhengdong Hu, Yi Yang

AI总结:

针对文本引导图像到图像扩散模型的来源溯源需求,本文提出ID²任务,构建了OriPID数据集,并提出基于预训练VAE嵌入线性变换的泛化方法,mAP较同类方法提升31.6%。

摘要:

文本引导的图像到图像扩散模型擅长根据文本提示转换图像,可实现精准且富有创意的视觉修改。然而,这项强大的技术可能被滥用于传播虚假信息、侵犯版权以及逃避内容追踪,这促使我们提出文本引导图像到图像扩散模型的来源识别(ID$^2$)任务,旨在检索给定转换后查询图像的原始图像。\nID$^2$任务的一种直接解决方案是训练专门的深度嵌入模型,从查询图像和参考图像中提取并比较特征。但由于不同扩散模型生成的图像存在视觉差异,这种基于相似度的方法在使用某一模型的图像训练、另一模型的图像测试时效果不佳,限制了其在实际场景中的应用。\n为解决所提出的ID$^2$任务面临的这一挑战,我们贡献了首个数据集和一种有理论保障的方法,二者均强调泛化能力。我们构建的OriPID数据集包含大量原始图像和引导提示,可用于在多种扩散模型上训练和测试潜在的识别模型。在方法部分,我们首先证明存在一种线性变换,能够最小化生成样本与其原始图像的预训练变分自编码器(VAE)嵌入之间的距离。随后我们证明,这种简单的线性变换可以在不同扩散模型间泛化。实验结果表明,所提方法取得了令人满意的泛化性能,显著优于基于相似度的方法(mAP提升31.6%),甚至优于那些带有泛化设计的方法。该项目可在https://id2icml.github.io获取。

英文摘要:

Text-guided image-to-image diffusion models excel in translating images based on textual prompts, allowing for precise and creative visual modifications. However, such a powerful technique can be misused for spreading misinformation, infringing on copyrights, and evading content tracing. This motivates us to introduce the task of origin IDentification for text-guided Image-to-image Diffusion models (ID$^2$), aiming to retrieve the original image of a given translated query. A straightforward solution to ID$^2$ involves training a specialized deep embedding model to extract and compare features from both query and reference images. However, due to visual discrepancy across generations produced by different diffusion models, this similarity-based approach fails when training on images from one model and testing on those from another, limiting its effectiveness in real-world applications. To solve this challenge of the proposed ID$^2$ task, we contribute the first dataset and a theoretically guaranteed method, both emphasizing generalizability. The curated dataset, OriPID, contains abundant Origins and guided Prompts, which can be used to train and test potential IDentification models across various diffusion models. In the method section, we first prove the existence of a linear transformation that minimizes the distance between the pre-trained Variational Autoencoder (VAE) embeddings of generated samples and their origins. Subsequently, it is demonstrated that such a simple linear transformation can be generalized across different diffusion models. Experimental results show that the proposed method achieves satisfying generalization performance, significantly surpassing similarity-based methods ($+31.6\%$ mAP), even those with generalization designs. The project is available at https://id2icml.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑