arXiv 2605.220722026-05-21cs.CLcs.CV

Faithful-MR1: 通过锚定和强化视觉注意力实现忠实的多模态推理

Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention

AMAP, Alibaba Group（阿里云实验室，阿里巴巴集团）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；Nanyang Technological University（南洋理工大学）

Changyuan Tian, Zhicong Lu, Huaxing Liu, Xiang Wang, Shuai Li, Yu Chen, Wenqian Lv, Zichuan Lin, Juncheng Diao, Deheng Ye

备注：: 20 pages, 7 figures, 3 tables. Preprint

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Faithful-MR1框架，通过锚定和强化视觉注意力解决多模态推理中的忠实性问题，提升模型在多模态基准上的表现。

摘要：

可验证奖励的强化学习（RLVR）已成为推动大语言模型复杂推理发展的有希望范式，最近的研究将其扩展到多模态大语言模型（MLLMs）。然而，这种转移带来了忠实性挑战：任务相关视觉证据的忠实感知以及在推理中忠实使用该证据，导致多模态基准上的不满意收益。具体而言，现有的感知监督通常基于文本描述而非原生的图像区域，且忠实使用被忽视，暴露出感知-推理断层，正确感知的证据在推理中被丢弃或矛盾。为弥合这些差距，我们提出Faithful-MR1，一个训练框架，通过锚定和强化视觉注意力来解决忠实多模态推理的两方面。锚定阶段将感知转化为一个显式的预推理子任务，监督专门的<Focus>标记的注意力直接针对图像区域，而不是通过文本描述。强化阶段通过反事实图像干预暴露忠实使用，奖励那些在视觉上因果重要的区域集中注意力的轨迹。广泛实验表明，Faithful-MR1在Qwen2.5-VL-Instruct 3B和7B架构上优于最近的多模态推理基线，同时使用大量训练数据。

英文摘要：

Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising paradigm for advancing complex reasoning in large language models, and recent work extends RLVR to multimodal large language models (MLLMs). This transfer, however, surfaces a faithfulness challenge: faithful perception of task-relevant visual evidence and faithful use of that evidence during reasoning, leading to unsatisfactory gains on multimodal benchmarks. Specifically, existing perception supervision often operates on textual descriptions rather than natively on image regions, and faithful use is largely overlooked, exposing the perception-reasoning disconnect where correctly perceived evidence is dropped or contradicted during reasoning. To close these gaps, we propose Faithful-MR1, a training framework that anchors and reinforces visual attention to address both halves of faithful multimodal reasoning. The Anchoring stage turns perception into an explicit pre-reasoning subtask, supervising a dedicated <Focus> token's attention directly against image regions rather than through textual descriptions. The Reinforcing stage exposes faithful use through counterfactual image intervention, rewarding answer-correct trajectories that concentrate visual attention where vision causally matters. Extensive experiments demonstrate that Faithful-MR1 outperforms recent multimodal reasoning baselines on both Qwen2.5-VL-Instruct 3B and 7B backbones while using substantially less training data.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。