通过对比学习与全局-局部相似性对比深度伪造扩散
Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities
AI总结：
本文提出专为深度伪造检测设计的新型嵌入空间CoDE，通过对比学习与全局-局部相似性进行训练，并构建含920万张扩散模型生成图像的数据集，实现了最先进的检测准确率与卓越的泛化能力。
摘要：
区分真实内容与由先进AI方法生成的内容已变得日益具有挑战性。以往研究主要针对伪造人脸的检测，而生成自然图像的识别问题最近才浮现。这促使近期探索采用如CLIP等基础视觉-语言模型的解决方案。然而，CLIP嵌入空间针对全局图文对齐进行优化，并非为深度伪造检测而设计，忽略了定制训练和局部图像特征的潜在益处。本研究提出CoDE（Contrastive Deepfake Embeddings，对比深度伪造嵌入），一种专为深度伪造检测设计的新型嵌入空间。CoDE通过对比学习训练，并额外强制执行全局-局部相似性。为支持模型训练，我们生成了一个综合数据集，专注于扩散模型生成的图像，包含使用四种不同生成器制作的920万张图像。实验结果表明，CoDE在新收集的数据集上实现了最先进的准确率，同时对未见过的图像生成器展现出卓越的泛化能力。源代码、训练模型和收集的数据集公开于：https://github.com/aimagelab/CoDE。
英文摘要：
Discerning between authentic content and that generated by advanced AI methods has become increasingly challenging. While previous research primarily addresses the detection of fake faces, the identification of generated natural images has only recently surfaced. This prompted the recent exploration of solutions that employ foundation vision-and-language models, like CLIP. However, the CLIP embedding space is optimized for global image-to-text alignment and is not inherently designed for deepfake detection, neglecting the potential benefits of tailored training and local image features. In this study, we propose CoDE (Contrastive Deepfake Embeddings), a novel embedding space specifically designed for deepfake detection. CoDE is trained via contrastive learning by additionally enforcing global-local similarities. To sustain the training of our model, we generate a comprehensive dataset that focuses on images generated by diffusion models and encompasses a collection of 9.2 million images produced by using four different generators. Experimental results demonstrate that CoDE achieves state-of-the-art accuracy on the newly collected dataset, while also showing excellent generalization capabilities to unseen image generators. Our source code, trained models, and collected dataset are publicly available at: https://github.com/aimagelab/CoDE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
