注意力引导的多模态大语言模型微调提升思维链推理能力
Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning
AI总结：
针对多模态大语言模型中思维链推理效果不佳的问题，提出注意力引导的微调目标Attentive-CoT，通过延迟答案承诺和维持视觉令牌访问来提升推理性能。
摘要：
思维链提示在多模态大语言模型中的有效性仍不确定：在多个视觉推理基准上，与直接提示相比，思维链提示常常降低性能。在本文中，我们对三个现代多模态大语言模型系列在不同模型规模下，针对需要逐步视觉证据的数据集进行了思维链行为的系统分析。我们的分析识别出两种反复出现的失败模式：过早的答案承诺和推理生成过程中有限的直接视觉令牌访问。我们进一步发现，标准的思维链式监督微调只能部分缓解这些问题，同时往往增加对文本先验的依赖并减少反事实视觉依赖。受这些发现的启发，我们提出了Attentive-CoT，一种注意力引导的微调目标，它鼓励思维链轨迹延迟答案承诺，同时维持持续的视觉令牌访问。Attentive-CoT可以插入任何思维链式监督微调训练中，无需架构更改。在六个多模态大语言模型上的三个视觉推理基准实验表明，Attentive-CoT相比标准微调提升了思维链性能。
英文摘要：
The effectiveness of Chain-of-Thought (CoT) prompting in Multimodal Large Language Models (MLLMs) remains uncertain: across several visual reasoning benchmarks, CoT prompting often degrades performance compared to direct prompting. In this paper, we provide a systematic analysis of CoT behavior in three modern MLLM families across model scales on datasets requiring step-wise visual evidence. Our analysis identifies two recurring failure modes: premature answer commitment and limited direct visual-token access during rationale generation. We further find that standard CoT-style Supervised Fine-Tuning (CoT-SFT) can mitigate these issues only partially, while often increasing reliance on textual priors and reducing counterfactual visual dependence. Motivated by these findings, we propose Attentive-CoT (Att-CoT), an attention-guided fine-tuning objective that encourages CoT trajectories to delay answer commitment while maintaining sustained visual-token access. Att-CoT can be plugged into any CoT-SFT training run without architectural changes. Experiments on three visual reasoning benchmarks across six MLLMs show that Att-CoT enhances CoT performance over standard fine-tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
