SPD-Faith Bench: 诊断和改进多模态大语言模型中的推理忠实性
SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Models
AI总结：
SPD-Faith Bench通过细粒度图像差异推理诊断并改进多模态大语言模型中的推理忠实性，揭示了感知盲区和感知-推理脱节的系统性失败模式，并提出SAGE框架提升视觉路由和推理与感知的一致性。
摘要：
链式推理被广泛用于提高多模态大语言模型（MLLMs）的可解释性，但生成的推理轨迹的忠实性仍然不明确。先前的工作主要集中在感知幻觉上，使推理层面的不忠实性未被充分探索。为了将忠实性与语言先验分离，我们引入了SPD-Faith Bench，这是一个基于细粒度图像差异推理的诊断基准，强制进行显式的视觉比较。在最先进的MLLMs上的评估揭示了两种系统性失败模式，感知盲区和感知-推理脱节。我们追溯这些失败到衰减的视觉注意力和残差流中的表征转移。基于此分析，我们提出了SAGE，一个无需训练的视觉证据校准框架，该框架提高了视觉路由并使推理与感知对齐。我们的结果强调了在响应正确性之外显式评估忠实性的重要性。我们的基准和代码可在https://github.com/Johanson-colab/SPD-Faith-Bench获取。
英文摘要：
Chain-of-Thought reasoning is widely used to improve the interpretability of multimodal large language models (MLLMs), yet the faithfulness of the generated reasoning traces remains unclear. Prior work has mainly focused on perceptual hallucinations, leaving reasoning level unfaithfulness underexplored. To isolate faithfulness from linguistic priors, we introduce SPD-Faith Bench, a diagnostic benchmark based on fine-grained image difference reasoning that enforces explicit visual comparison. Evaluations on state-of-the-art MLLMs reveal two systematic failure modes, perceptual blindness and perception-reasoning dissociation. We trace these failures to decaying visual attention and representation shifts in the residual stream. Guided by this analysis, we propose SAGE, a train-free visual evidence-calibrated framework that improves visual routing and aligns reasoning with perception. Our results highlight the importance of explicitly evaluating faithfulness beyond response correctness. Our benchmark and codes are available at https://github.com/Johanson-colab/SPD-Faith-Bench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
