通过面向基础模型的面部组件引导适配实现更通用的基于视频的 Deepfake 检测
Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model
AI总结：
针对基于视频的 Deepfake 检测在未见伪造样本上泛化不足的问题，提出基于侧网络的 CLIP 解码器并引入 FCG，提升了泛化性、数据与参数效率及鲁棒性。
摘要：
生成模型已能够创建高度逼真的面部合成图像，因其可能被滥用而引发了严重担忧。尽管 Deepfake 检测领域发展迅速，但开发高效方法以利用基础模型提升对未见伪造样本的泛化能力仍然具有挑战性。为应对这一挑战，我们提出一种新颖的基于侧网络的解码器，该解码器使用 CLIP 图像编码器提取空间和时间线索，以实现通用的基于视频的 Deepfake 检测。此外，我们引入面部组件引导（Facial Component Guidance，FCG），通过鼓励模型关注关键面部区域来增强空间学习的泛化能力。通过利用视觉-语言基础模型的通用特征，我们的方法在具有挑战性的 Deepfake 数据集上展现出良好的泛化能力，同时在训练数据效率、参数效率和模型鲁棒性方面表现出优势。
英文摘要：
Generative models have enabled the creation of highly realistic facial-synthetic images, raising significant concerns due to their potential for misuse. Despite rapid advancements in the field of deepfake detection, developing efficient approaches to leverage foundation models for improved generalizability to unseen forgery samples remains challenging. To address this challenge, we propose a novel side-network-based decoder that extracts spatial and temporal cues using the CLIP image encoder for generalized video-based Deepfake detection. Additionally, we introduce Facial Component Guidance (FCG) to enhance spatial learning generalizability by encouraging the model to focus on key facial regions. By leveraging the generic features of a vision-language foundation model, our approach demonstrates promising generalizability on challenging Deepfake datasets while also exhibiting superiority in training data efficiency, parameter efficiency, and model robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
