取证适配器：释放CLIP用于通用人脸伪造检测
Forensics Adapter: Unleashing CLIP for Generalizable Face Forgery Detection
AI总结：
研究旨在将CLIP转变为人脸伪造检测器，提出取证适配器及扩展方法取证适配器++。通过适配器学习伪造痕迹，用交互策略增强视觉令牌，实现通用性。方法参数少性能优，扩展方法进一步提升性能，为相关检测提供基线。
摘要：
我们描述了取证适配器，这是一种适配器网络，旨在将CLIP转变为一个有效且通用的人脸伪造检测器。尽管CLIP用途广泛，但将其用于人脸伪造检测并非易事，因为伪造相关知识与大量无关知识交织在一起。现有方法仅将CLIP用作特征提取器，缺乏特定任务的适配，限制了其有效性。为此，我们引入一个适配器，在特定任务目标的引导下学习人脸伪造痕迹，即伪造人脸特有的融合边界。然后我们通过一种专门的交互策略增强CLIP视觉令牌，该策略在CLIP和适配器之间传递知识。由于适配器与CLIP并行，其通用性得到高度保留，自然确保了在人脸伪造检测中的强大通用性。我们的方法仅用570万个可训练参数，就在六个标准数据集上取得了卓越性能。此外，我们还描述了取证适配器++，这是一种扩展方法，通过新提出的伪造感知提示学习策略纳入文本模态。此扩展比原始取证适配器性能进一步提高1.3%。我们相信所提出的方法可作为未来基于CLIP的人脸伪造检测方法的基线。代码已在该https网址发布。
英文摘要：
We describe Forensics Adapter, an adapter network designed to transform CLIP into an effective and generalizable face forgery detector. Although CLIP is highly versatile, adapting it for face forgery detection is non-trivial as forgery-related knowledge is entangled with a wide range of unrelated knowledge. Existing methods treat CLIP merely as a feature extractor, lacking task-specific adaptation, which limits their effectiveness. To address this, we introduce an adapter to learn face forgery traces -- the blending boundaries unique to forged faces, guided by task-specific objectives. Then we enhance the CLIP visual tokens with a dedicated interaction strategy that communicates knowledge across CLIP and the adapter. Since the adapter is alongside CLIP, its versatility is highly retained, naturally ensuring strong generalizability in face forgery detection. {With only $\textbf{5.7M}$ trainable parameters, our method achieves superior performance across six standard datasets.} Additionally, we describe Forensics Adapter++, an extended method that incorporates textual modality via a newly proposed forgery-aware prompt learning strategy. This extension leads to a further $\textbf{1.3\%}$ performance boost over the original Forensics Adapter. We believe the proposed methods can serve as a baseline for future CLIP-based face forgery detection methods. The code has been released at https://github.com/OUC-VAS/ForensicsAdapter.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
