面向文本模态的图像特征提取用于检测基于扩散的深度伪造
Text Modality Oriented Image Feature Extraction for Detecting Diffusion-based DeepFake
AI总结：
针对扩散模型生成的深度伪造图像，提出面向文本模态的特征提取方法TOFE，利用文本嵌入同时捕获低级与高级特征，在十种扩散类型上验证了有效性。
摘要：
扩散方法的广泛使用使得按需生成高度逼真的图像成为可能，从而对在线信息的完整性和安全性构成重大风险，并凸显了深度伪造检测的必要性。我们对传统图像编码器提取的特征进行分析后发现，低级特征和高级特征在识别由不同扩散方法生成的深度伪造图像方面各有独特优势。受此发现启发，我们旨在开发一种能够同时捕获低级和高级特征的有效表示，以检测基于扩散的深度伪造。为解决该问题，我们提出了一种面向文本模态的特征提取方法，称为TOFE。具体而言，对于给定的目标图像，我们发现其表示是对应的文本嵌入，该文本嵌入可引导特定文本到图像模型生成目标图像。在十种扩散类型上进行的实验证明了我们提出方法的有效性。
英文摘要：
The widespread use of diffusion methods enables the creation of highly realistic images on demand, thereby posing significant risks to the integrity and safety of online information and highlighting the necessity of DeepFake detection. Our analysis of features extracted by traditional image encoders reveals that both low-level and high-level features offer distinct advantages in identifying DeepFake images produced by various diffusion methods. Inspired by this finding, we aim to develop an effective representation that captures both low-level and high-level features to detect diffusion-based DeepFakes. To address the problem, we propose a text modality-oriented feature extraction method, termed TOFE. Specifically, for a given target image, the representation we discovered is a corresponding text embedding that can guide the generation of the target image with a specific text-to-image model. Experiments conducted across ten diffusion types demonstrate the efficacy of our proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
