arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

通过生成顺序干预评估解释驱动的视觉-语言推理

Evaluating Explanation-Driven Vision-Language Reasoning via Generation Order Interventions

Siting Liang, Luca Rippe, Omar Adjali, Daniel Sonntag

arXiv 2609.29496首次发表:更新:

发表机构

German Research Center for Artificial Intelligence; University of Oldenburg(德国人工智能研究中心; 奥尔登堡大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

本研究通过控制生成顺序干预,系统评估了解释与预测的因果关联,发现模型规模是理由优先推理的关键,而答案优先更稳定,二者共同影响推理忠实度。

AI 中文摘要

自然语言解释生成是揭示和评估视觉-语言推理的关键机制。以往关于解释驱动的视觉-语言模型的工作主要遵循事后(答案优先)范式,这隐含地表明监督理由可以反映底层推理过程。相比之下,现代大型视觉-语言模型日益表现出理由优先的生成倾向,这与结构化、逐步推理更为一致。在本工作中,我们在受控实验设置下系统评估解释是否在单个生成步骤内与模型预测存在因果关联,明确排除了知识密集型问答、视觉蕴含和组合式基础基准中不必要的思维链或其他中间推理过程。我们发现,较大模型成为大规模可靠支持理由优先推理的先决条件。然而,答案优先生成在结构化输出中较不易出现格式相关错误。总体而言,解释顺序、模型规模和预训练知识、任务特定微调以及任务结构共同影响预测准确性和推理忠实度。

英文摘要

Natural language explanation generation serves as a key mechanism for exposing and evaluating vision-language reasoning. Prior work on explanation-driven vision-language models predominantly follows a post-hoc (answer-first) paradigm, implicitly suggesting that supervised rationales can reflect underlying reasoning processes. In contrast, modern large vision-language models increasingly exhibit a rationale-first generation tendency, which more closely aligns with structured, stepwise reasoning. In this work, we systematically evaluate whether explanations are causally tied to model predictions within a single generation step under a controlled experimental setup, explicitly eliminating unnecessary chain-of-thought or other intermediate reasoning processes across knowledge-intensive QA, visual entailment, and compositional grounding benchmarks. We find that larger models emerge as a prerequisite for reliably supporting rationale-first reasoning at scale. However, answer-first generation is less prone to format-related errors in structured output. Overall, explanation ordering, model scale and pre-training knowledge, task-specific fine-tuning, and task structure jointly influence both prediction accuracy and reasoning faithfulness.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑