arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2504.04470cs.CV
收藏

基于内容感知复合提示工程的人脸反欺骗领域泛化

Domain Generalization for Face Anti-spoofing via Content-aware Composite Prompt Engineering

School of Computer Science, Hefei University of Technology(合肥工业大学计算机学院);State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);School of Electronic Information Engineering, Taiyuan University of Technology(太原理工大学电子信息工程学院);School of Computer Science and Engineering, Faculty of Innovation Engineering, Macau University of Science and Technology(澳门科技大学创新工程学院计算机科学与工程学院);School of Cyber Science and Engineering, Wuhan University(武汉大学网络空间安全学院)

Jiabao Guo, Ajian Liu, Yunfeng Diao, Jin Zhang, Hui Ma, Bo Zhao, Richang Hong, Meng Wang

AI总结:

本文提出内容感知复合提示工程(CCPE),通过结合大语言模型固有内容提示与Q-Former可学习内容提示,并引入跨模态引导模块,解决人脸反欺骗领域泛化中领域干扰问题,取得最先进性能。

摘要:

人脸反欺骗(FAS)中领域泛化(DG)面临的挑战在于领域特定信号对细微欺骗线索的显著干扰。近年来,一些基于CLIP的算法通过调整视觉分类器的权重来缓解这种干扰。然而,我们对这种类别级提示工程的分析表明,其在DG FAS中存在两个不足:(1)人脸类别(如真实或欺骗)对CLIP模型而言没有语义,难以学习准确的类别描述;(2)单一形式的提示无法刻画各种类型的欺骗。在本工作中,我们不再使用类别级提示,而是提出了一种新颖的内容感知复合提示工程(CCPE),它生成实例级复合提示,包括固定模板和可学习提示。具体来说,我们的CCPE从两个分支构建内容感知提示:(1)固有内容提示明确受益于基于指令的大语言模型(LLM)提供的丰富迁移知识;(2)可学习内容提示通过Q-Former隐式提取最具信息量的视觉内容。此外,我们设计了一个跨模态引导模块(CGM),动态调整单模态特征以进行融合,从而实现更好的泛化FAS。最后,我们的CCPE在多个跨领域实验中验证了其有效性,并取得了最先进(SOTA)的结果。

英文摘要:

The challenge of Domain Generalization (DG) in Face Anti-Spoofing (FAS) is the significant interference of domain-specific signals on subtle spoofing clues. Recently, some CLIP-based algorithms have been developed to alleviate this interference by adjusting the weights of visual classifiers. However, our analysis of this class-wise prompt engineering suffers from two shortcomings for DG FAS: (1) The categories of facial categories, such as real or spoof, have no semantics for the CLIP model, making it difficult to learn accurate category descriptions. (2) A single form of prompt cannot portray the various types of spoofing. In this work, instead of class-wise prompts, we propose a novel Content-aware Composite Prompt Engineering (CCPE) that generates instance-wise composite prompts, including both fixed template and learnable prompts. Specifically, our CCPE constructs content-aware prompts from two branches: (1) Inherent content prompt explicitly benefits from abundant transferred knowledge from the instruction-based Large Language Model (LLM). (2) Learnable content prompts implicitly extract the most informative visual content via Q-Former. Moreover, we design a Cross-Modal Guidance Module (CGM) that dynamically adjusts unimodal features for fusion to achieve better generalized FAS. Finally, our CCPE has been validated for its effectiveness in multiple cross-domain experiments and achieves state-of-the-art (SOTA) results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑