多模态事实级归因用于可验证推理
Multimodal Fact-Level Attribution for Verifiable Reasoning
AI总结：
本文提出MuRGAt基准，用于评估多模态事实归因能力，要求模型在复杂推理中生成明确推理和精确引用，揭示强模型在正确推理下仍易产生幻觉，且增加推理深度或结构化归因会降低准确性。
摘要：
多模态大语言模型（MLLMs）越来越多地用于涉及多步推理和长文本生成的现实任务，其中可靠性要求将模型输出接地于异构输入源并验证个别事实性声明。然而，现有多模态接地基准和评估方法专注于简化、基于观察的场景或有限模态，无法评估复杂多模态推理中的归因能力。我们引入MuRGAt（多模态推理与接地归因），一个用于评估事实级多模态归因的基准，适用于需要超越直接观察的推理场景。给定涵盖视频、音频和其他模态的输入，MuRGAt要求模型生成带有明确推理和精确引用的答案，其中每个引用指定模态和时间片段。为了实现可靠评估，我们引入了一个自动评估框架，其与人类判断高度相关。与人类和自动化评分的基准测试显示，即使强大的MLLMs在正确推理下仍频繁产生幻觉引用。此外，我们观察到关键权衡：增加推理深度或强制结构化接地往往会降低准确性，凸显了内部推理与可验证归因之间的显著差距。
英文摘要：
Multimodal large language models (MLLMs) are increasingly used for real-world tasks involving multi-step reasoning and long-form generation, where reliability requires grounding model outputs in heterogeneous input sources and verifying individual factual claims. However, existing multimodal grounding benchmarks and evaluation methods focus on simplified, observation-based scenarios or limited modalities and fail to assess attribution in complex multimodal reasoning. We introduce MuRGAt (Multimodal Reasoning with Grounded Attribution), a benchmark for evaluating fact-level multimodal attribution in settings that require reasoning beyond direct observation. Given inputs spanning video, audio, and other modalities, MuRGAt requires models to generate answers with explicit reasoning and precise citations, where each citation specifies both modality and temporal segments. To enable reliable assessment, we introduce an automatic evaluation framework that strongly correlates with human judgments. Benchmarking with human and automated scores reveals that even strong MLLMs frequently hallucinate citations despite correct reasoning. Moreover, we observe a key trade-off: increasing reasoning depth or enforcing structured grounding often degrades accuracy, highlighting a significant gap between internal reasoning and verifiable attribution.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
