C2P-CLIP：在 CLIP 中注入类别通用提示以增强 Deepfake 检测中的泛化能力
C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detection
AI总结：
该研究揭示 CLIP 依赖相似概念识别 deepfake，并提出 C2P-CLIP，通过类别通用提示向图像编码器注入类别概念，在无额外测试参数下将准确率提升 12.41%。
摘要：
本工作聚焦 AIGC 检测，旨在开发能够识别各类伪造图像的通用检测器。近期研究发现，CLIP 等大型预训练模型结合线性分类器，可有效实现具有泛化性的 deepfake 检测。然而，仍有两个关键问题尚未解决：1）理解为什么 CLIP 特征通过线性分类器就能在 deepfake 检测中有效；2）挖掘 CLIP 的检测潜力。本研究通过将其检测特征解码为文本并进行词频分析，深入探究 CLIP 检测能力背后的机制。我们的发现表明，CLIP 通过识别相似概念来检测 deepfakes（图 \ ef{fig:fig1} a）。基于这一洞见，我们提出了 Category Common Prompt CLIP，称为 C2P-CLIP，它将类别通用提示集成到文本编码器中，以把类别相关概念注入图像编码器，从而提升检测性能（图 \ ef{fig:fig1} b）。与原始 CLIP 相比，我们的方法在检测准确率上实现了 12.41% 的提升，并且在测试过程中不引入额外参数。在两个广泛使用的数据集上开展的综合实验涵盖 20 个生成模型，验证了所提方法的有效性，并展示了最先进性能。代码可在 https://github.com/chuangchuangtan/C2P-CLIP-DeepfakeDetection 获取。
英文摘要：
This work focuses on AIGC detection to develop universal detectors capable of identifying various types of forgery images. Recent studies have found large pre-trained models, such as CLIP, are effective for generalizable deepfake detection along with linear classifiers. However, two critical issues remain unresolved: 1) understanding why CLIP features are effective on deepfake detection through a linear classifier; and 2) exploring the detection potential of CLIP. In this study, we delve into the underlying mechanisms of CLIP's detection capabilities by decoding its detection features into text and performing word frequency analysis. Our finding indicates that CLIP detects deepfakes by recognizing similar concepts (Fig. \ref{fig:fig1} a). Building on this insight, we introduce Category Common Prompt CLIP, called C2P-CLIP, which integrates the category common prompt into the text encoder to inject category-related concepts into the image encoder, thereby enhancing detection performance (Fig. \ref{fig:fig1} b). Our method achieves a 12.41\% improvement in detection accuracy compared to the original CLIP, without introducing additional parameters during testing. Comprehensive experiments conducted on two widely-used datasets, encompassing 20 generation models, validate the efficacy of the proposed method, demonstrating state-of-the-art performance. The code is available at \url{https://github.com/chuangchuangtan/C2P-CLIP-DeepfakeDetection}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
