arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.16629cs.CVcs.AI
收藏

基于MLLM的文本解释用于面部比较

MLLM-based Textual Explanations for Face Comparison

Michigan State University(密歇根州立大学)

Redwan Sony, Anil K Jain, Arun Ross

AI总结:

本文研究了MLLM在无约束面部验证任务中生成解释的可靠性,发现其解释常依赖不可验证的面部属性,并提出基于似然比的框架评估解释可信度。

摘要:

多模态大语言模型(MLLMs)最近被提出用于生成面部识别决策的自然语言解释。尽管这些解释有助于人类可解释性,但其在无约束面部图像上的可靠性仍缺乏研究。本文系统分析了MLLM在挑战性IJB-S数据集上的面部验证任务中生成的解释,特别关注极端姿态变化和监控图像。我们的结果表明,即使MLLM做出正确的验证决策,伴随的解释也经常依赖于不可验证或虚构的面部属性,这些属性没有视觉证据支持。我们进一步研究了将传统面部识别系统的信息(即分数和决策)与输入图像结合的效果。尽管此类信息提高了分类验证性能,但并不始终导致忠实的解释。为了评估解释超越决策准确性的能力,我们引入了一个基于似然比的框架,该框架衡量文本解释的证据强度。我们的发现突显了当前MLLM在可解释性面部识别中的基本限制,并强调了在生物识别应用中需要对可靠和可信的解释进行原则性评估的必要性。代码可在https://github.com/redwankarimsony/LR-MLLMFR-Explainability获取。

英文摘要:

Multimodal Large Language Models (MLLMs) have recently been proposed as a means to generate natural-language explanations for face recognition decisions. While such explanations facilitate human interpretability, their reliability on unconstrained face images remains underexplored. In this work, we systematically analyze MLLM-generated explanations for the unconstrained face verification task on the challenging IJB-S dataset, with a particular focus on extreme pose variation and surveillance imagery. Our results show that even when MLLMs produce correct verification decisions, the accompanying explanations frequently rely on non-verifiable or hallucinated facial attributes that are not supported by visual evidence. We further study the effect of incorporating information from traditional face recognition systems, viz., scores and decisions, alongside the input images. Although such information improves categorical verification performance, it does not consistently lead to faithful explanations. To evaluate the explanations beyond decision accuracy, we introduce a likelihood-ratio-based framework that measures the evidential strength of textual explanations. Our findings highlight fundamental limitations of current MLLMs for explainable face recognition and underscore the need for a principled evaluation of reliable and trustworthy explanations in biometric applications. Code is available at https://github.com/redwankarimsony/LR-MLLMFR-Explainability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑