MLLMs真的能看见吗：在多模态大语言模型中强化视觉注意力
Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs
AI总结：
SAYO通过强化学习框架引入区域级视觉注意力奖励，提升多模态大语言模型的视觉聚焦能力，从而在多种推理和感知任务中取得性能提升。
摘要：
尽管链式推理（CoT）推理已显著提升了多模态大语言模型（MLLMs）在复杂推理任务上的表现，但现有方法主要依赖于长文本推理轨迹，并提供有限的机制来学习稳定的视觉注意力策略。我们的分析显示，当前的MLLMs存在视觉聚焦薄弱的问题：早期阶段的视觉偏差在后续推理中很少被纠正，导致误差传播和推理失败。我们认为这一限制源于训练过程中对视觉注意力的信用分配不足。为了解决这一问题，我们提出了SAYO，一种通过强化学习（RL）框架训练的视觉推理模型，引入了基于区域级视觉注意力的奖励。该奖励明确将优化信号与视觉基础推理步骤对齐，使模型能够学习更可靠的注意力行为。在多个多模态基准上的广泛实验表明，SAYO在多样化的推理和感知任务上 consistently 提高了性能。
英文摘要：
While chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks, existing approaches largely rely on long textual reasoning trajectories and provide limited mechanisms for learning stable visual attention policies. Our analysis shows that current MLLMs exhibit weak visual focus: early-stage visual misalignment is rarely corrected during subsequent reasoning, leading to error propagation and failed inferences. We argue that this limitation stems from inadequate credit assignment for visual attention during training. To address this issue, we propose SAYO, a visual reasoning model trained with a reinforcement learning (RL) framework that introduces a region-level visual attention-based reward. This reward explicitly aligns optimization signals with visually grounded reasoning steps, enabling the model to learn more reliable attention behaviors. Extensive experiments across multiple multimodal benchmarks demonstrate that SAYO consistently improves performance on diverse reasoning and perception tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
