MM-CoT：一种用于探测多模态模型中视觉链式推理的基准测试
MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models
AI总结：
MM-CoT是一种用于评估多模态模型视觉链式推理能力的基准测试，通过验证推理链的视觉一致性和逻辑一致性，揭示生成模型在真实推理上的不足。
摘要：
进行链式推理（CoT）的能力标志着多模态模型（MMs）的重要里程碑，使它们能够解决复杂的视觉推理问题。然而，一个关键问题仍然存在：这种推理是否真正基于视觉证据并且逻辑上一致？现有的基准测试强调生成但忽视验证，即评估推理链是否在视觉上一致且逻辑上有效的能力。为填补这一空白，我们引入MM-CoT，一个专门设计用于探测多模态模型中链式推理的视觉基础和逻辑一致性的诊断基准测试。与生成自由形式解释不同，模型必须选择唯一一个满足两个正交约束的事件链：（i）视觉一致性，确保所有步骤都基于可观察的证据；（ii）逻辑一致性，确保因果和常识的有效性。对抗性干扰项被设计成违反其中一个约束，暴露不同的推理失败。我们评估了领先的视觉-语言模型在MM-CoT上的表现，并发现即使是最先进的系统也难以应对，揭示了生成流畅性与真实推理忠实性之间的显著差距。MM-CoT与现有基准测试的相关性较低，证实了它测量了一种独特的视觉基础和逻辑推理的组合。该基准测试为开发未来能够真实且一致地在视觉世界中推理的模型提供了基础。
英文摘要：
The ability to perform Chain-of-Thought (CoT) reasoning marks a major milestone for multimodal models (MMs), enabling them to solve complex visual reasoning problems. Yet a critical question remains: is such reasoning genuinely grounded in visual evidence and logically coherent? Existing benchmarks emphasize generation but neglect verification, i.e., the capacity to assess whether a reasoning chain is both visually consistent and logically valid. To fill this gap, we introduce MM-CoT, a diagnostic benchmark specifically designed to probe the visual grounding and logical coherence of CoT reasoning in MMs. Instead of generating free-form explanations, models must select the sole event chain that satisfies two orthogonal constraints: (i) visual consistency, ensuring all steps are anchored in observable evidence, and (ii) logical coherence, ensuring causal and commonsense validity. Adversarial distractors are engineered to violate one of these constraints, exposing distinct reasoning failures. We evaluate leading vision-language models on MM-CoT and find that even the most advanced systems struggle, revealing a sharp discrepancy between generative fluency and true reasoning fidelity. MM-CoT shows low correlation with existing benchmarks, confirming that it measures a unique combination of visual grounding and logical reasoning. This benchmark provides a foundation for developing future models that reason not just plausibly, but faithfully and coherently within the visual world.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
