arXiv 2509.247762025-09-29cs.CVcs.AI

VTPerception-R1：通过显式视觉与文本感知基础增强多模态推理

VTPerception-R1: Enhancing Multimodal Reasoning via Explicit Visual and Textual Perceptual Grounding

Fudan University（复旦大学）；Shanghai AI Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）；Shenzhen University（深圳大学）；University of Science and Technology of China（中国科学技术大学）

Yizhuo Ding, Mingkang Chen, Zhibang Feng, Tong Xiao, Wanying Qu, Wenqi Shao, Yanwei Fu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对 MLLM 推理缺乏感知证据支撑的问题，本文提出两阶段框架 VTPerception-R1，通过感知增强微调及含视觉、文本与一致性奖励的强化学习，提升多模态推理的准确性与鲁棒性。

摘要：

多模态大语言模型（MLLMs）往往难以将推理建立在感知证据之上。我们在四个多模态基准和两个 MLLM 上，对显式、隐式、视觉和文本等感知策略进行了系统研究。研究结果表明，显式感知（尤其是与文本线索结合时）能够持续带来最佳提升，对较小模型而言效果尤为明显。基于这一洞见，我们提出 VTPerception-R1，这是一个将感知与推理解耦的统一两阶段框架。阶段 1 引入感知增强微调，阶段 2 应用感知感知强化学习，并采用新颖的视觉、文本和一致性奖励。实验表明，VTPerception-R1 在多种任务上显著提升了推理准确性和鲁棒性，为基于感知基础的多模态推理提供了可扩展且可审计的解决方案。我们的代码可在 https://github.com/yizhuoDi/VTPerceprion-R1 获取。

英文摘要：

Multimodal large language models (MLLMs) often struggle to ground reasoning in perceptual evidence. We present a systematic study of perception strategies-explicit, implicit, visual, and textual-across four multimodal benchmarks and two MLLMs. Our findings show that explicit perception, especially when paired with textual cues, consistently yields the best improvements, particularly for smaller models. Based on this insight, we propose VTPerception-R1, a unified two-stage framework that decouples perception from reasoning. Stage 1 introduces perception-augmented fine-tuning, and Stage 2 applies perception-aware reinforcement learning with novel visual, textual, and consistency rewards. Experiments demonstrate that VTPerception-R1 significantly improves reasoning accuracy and robustness across diverse tasks, offering a scalable and auditable solution for perception-grounded multimodal reasoning. Our code is available at: https://github.com/yizhuoDi/VTPerceprion-R1.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。