弥合感知与推理：用于多模态大语言模型中RLVR的标记重加权
Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
AI总结：
本文提出Token-Reweighting策略，通过动态重加权多模态大语言模型中的感知与推理标记，提升RLVR性能，实现视觉 grounding 和推理的协同优化。
摘要：
将可验证奖励强化学习（RLVR）扩展到多模态大语言模型（MLLMs）面临根本性挑战：其响应本质上交织了感知相关标记（承载视觉内容）与推理相关标记（构建推理链）。这些标记类型体现了不同但相互依赖的能力——视觉 grounding 和符号推理——使得孤立优化不足。通过标记层面的经验分析，我们证明优化感知或推理单独标记始终比完整优化表现差，凸显其内在耦合。为此，我们提出一种插件式Token-Reweighting（ToR）策略，通过识别两种类型的关键标记并动态重加权来显式建模这种依赖性。在现有方法（如GRPO和DAPO）之上应用ToR，在多个多模态推理基准上实现了稳定的性能提升，同时在准确的视觉 grounding 和连贯的推理方面达到最先进的性能。
英文摘要：
Extending Reinforcement Learning with Verifiable Rewards (RLVR) to multimodal large language models (MLLMs) faces a fundamental challenge: their responses inherently interleave perception-related tokens, which ground visual content, with reasoning-related tokens, which construct reasoning chains. These token types instantiate distinct yet interdependent capacities -- visual grounding and symbolic reasoning -- making isolated optimization insufficient. Through token-level empirical analysis, we demonstrate that optimizing either perception- or reasoning-only tokens consistently underperforms full optimization, underscoring their inherent coupling. To address this, we propose a plug-and-play Token-Reweighting (ToR) strategy that explicitly models this interdependence by identifying critical tokens of both types and dynamically reweighting them during RLVR training. Applied on top of existing methods (e.g., GRPO and DAPO), ToR delivers consistent performance gains across multiple multi-modal reasoning benchmarks, achieving state-of-the-art performance with both accurate visual grounding and coherent reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
