arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.15461cs.CV
收藏

开放多模态大语言模型中的涌现形变攻击检测

Emergent Morphing Attack Detection in Open Multi-modal Large Language Models

Faculty of Electrical Engineering, University of Ljubljana(卢布尔雅那大学电气工程学院)

Marija Ivanovska, Vitomir Štruc

AI总结:

本文提出利用开源多模态大语言模型进行零样本人脸形变攻击检测,通过实验表明其在无微调情况下具备优异的判别能力,性能超越传统基线23%。

摘要:

人脸形变攻击威胁生物识别验证,但大多数形变攻击检测(MAD)系统需要任务特定的训练且对未见过的攻击类型泛化能力差。同时,开源多模态大语言模型(MLLMs)已显示出强大的视觉-语言推理能力,但其在生物识别取证中的潜力仍待探索。本文首次系统性地对开源MLLMs进行了单图像MAD的零样本评估,使用公开可用的权重和标准化、可重复的协议。在多样化的形变技术中,许多MLLMs在无任何微调或领域适应的情况下展示了非平凡的判别能力,且LLaVA1.6-Mistral-7B实现了最先进的性能,以等错误率(EER)指标超越了高度竞争的任务特定MAD基线至少23%。结果表明,多模态预训练可以隐式编码细粒度面部不一致,这些不一致指示形变伪影,从而实现零样本取证灵敏度。我们的发现将开源MLLMs定位为可重复、可解释且竞争性的生物识别安全和取证图像分析的基础。这种涌现能力也突显了通过针对性微调或轻量级适应开发最先进MAD系统的新机会,进一步提高准确性和效率的同时保持可解释性。为支持未来研究,所有代码和评估协议将在发表时发布。

英文摘要:

Face morphing attacks threaten biometric verification, yet most morphing attack detection (MAD) systems require task-specific training and generalize poorly to unseen attack types. Meanwhile, open-source multimodal large language models (MLLMs) have demonstrated strong visual-linguistic reasoning, but their potential in biometric forensics remains underexplored. In this paper, we present the first systematic zero-shot evaluation of open-source MLLMs for single-image MAD, using publicly available weights and a standardized, reproducible protocol. Across diverse morphing techniques, many MLLMs show non-trivial discriminative ability without any fine-tuning or domain adaptation, and LLaVA1.6-Mistral-7B achieves state-of-the-art performance, surpassing highly competitive task-specific MAD baselines by at least 23% in terms of equal error rate (EER). The results indicate that multimodal pretraining can implicitly encode fine-grained facial inconsistencies indicative of morphing artifacts, enabling zero-shot forensic sensitivity. Our findings position open-source MLLMs as reproducible, interpretable, and competitive foundations for biometric security and forensic image analysis. This emergent capability also highlights new opportunities to develop state-of-the-art MAD systems through targeted fine-tuning or lightweight adaptation, further improving accuracy and efficiency while preserving interpretability. To support future research, all code and evaluation protocols will be released upon publication.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑