GridVQA-X: 评估多模态可解释性方法的框架
GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods
AI总结：
提出GridVQA-X诊断框架，通过合成数据生成数学保证的解释，并训练纯推理与捷径依赖的配对模型，揭示现有可解释性方法无法区分真实跨模态推理与浅层捷径。
摘要：
随着视觉-语言模型的不断发展，其预测结果对相关利益方具有可解释性变得至关重要。然而，可解释性领域并未跟上多模态发展的步伐。尽管最近的多模态可解释人工智能（MxAI）方法生成解释以归因不同模态之间的交互，但当前的评估协议缺乏区分真正跨模态推理（例如，空间组合）与浅层跨模态捷径（例如，词袋属性匹配）所需的地面真相。目前尚不清楚MxAI方法是否忠实地捕捉了协同交互，或者仅仅是对作为简单特征检测器的模型进行推理幻觉。在本文中，我们介绍了GridVQA-X，这是第一个专门设计用于评估跨模态可解释性的诊断框架。与自然数据集不同，GridVQA-X利用封闭世界合成逻辑生成独特的、数学上保证的解释。我们利用这个受控环境，在相同的架构上训练配对的真实模型：$M_{\ ext{pure}}$，学习稳健的空间关系推理，以及$M_{\ ext{spur}}$，结构上被迫依赖跨模态捷径。这种行为差异创建了一个严格的测试平台：一个忠实的解释器必须为每个模型报告不同的推理路径。我们的发现表明，广泛使用的方法无法区分依赖真正空间关系推理的模型和利用跨模态捷径的模型，突显了在捕捉真正跨模态协同方面的关键差距，并错误地表示了多模态模型实际如何做出决策。
英文摘要：
With the increasing development of Vision-Language Models, it becomes imperative that their predictions are readily explainable to relevant stakeholders. However, the field of explainability has not kept pace with the multimodal surge. While recent Multimodal Explainable AI (MxAI) methods generate explanations to attribute the interaction between different modalities, current evaluation protocols lack the ground truth required to distinguish between true cross-modal reasoning (e.g., spatial composition) and shallow cross-modal shortcuts (e.g., Bag-of-Words attribute matching). It remains unknown whether MxAI methods faithfully capture synergistic interactions or merely hallucinate reasoning on models acting as simple feature detectors. In this paper, we introduce GridVQA-X, the first diagnostic framework specifically designed to evaluate cross-modal explainability. Unlike natural datasets, GridVQA-X leverages a closed-world synthesis logic to generate unique, mathematically guaranteed explanations. We utilize this controlled environment to train paired ground-truth models on identical architectures: $M_{\text{pure}}$, which learns robust spatial-relational reasoning and $M_{\text{spur}}$, which is structurally forced to rely on cross-modal shortcuts. This behavioral divergence creates a rigorous testbed: a faithful explainer must report distinct reasoning pathways for each model. Our findings reveal that widely used methods fail to distinguish between models relying on genuine spatial-relational reasoning and those exploiting cross-modal shortcuts, highlighting a critical gap in capturing true cross-modal synergy and misrepresenting how multimodal models actually make decisions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
