不止于最终答案：提升视觉提取和逻辑一致性的视觉语言模型
More Than the Final Answer: Improving Visual Extraction and Logical Consistency in Vision-Language Models
AI总结：
PeRL-VL通过解耦框架提升视觉语言模型的视觉提取和推理一致性，实现Pass@1准确率提升。
摘要：
基于可验证奖励的强化学习（RLVR）最近已被扩展到视觉语言模型（VLMs），以促使长链多模态推理。然而，训练后的VLMs仍然存在两个持续的失败模式：不准确的视觉提取（缺失或幻觉细节）和逻辑不一致的推理链，这主要是因为可验证信号仅监督最终答案。我们提出了PeRL-VL（面向视觉语言模型的感知和推理学习），一个解耦框架，分别在RLVR基础上改进视觉感知和文本推理。在感知方面，PeRL-VL引入了一个基于VLM的描述奖励，对模型自动生成的图像描述进行评分，以评估其忠实性和充分性。在推理方面，PeRL-VL在逻辑丰富的推理链数据上添加了一个纯文本的推理SFT阶段，增强推理的连贯性和逻辑一致性，而不依赖于视觉。在多样化的多模态基准测试中，PeRL-VL将平均Pass@1准确率从63.3%（基础Qwen2.5-VL-7B）提升到68.8%，优于标准RLVR、纯文本推理SFT和从GPT-4o进行的简单多模态蒸馏。
英文摘要：
Reinforcement learning from verifiable rewards (RLVR) has recently been extended from text-only LLMs to vision-language models (VLMs) to elicit long-chain multimodal reasoning. However, RLVR-trained VLMs still exhibit two persistent failure modes: inaccurate visual extraction (missing or hallucinating details) and logically inconsistent chains-of-thought, largely because verifiable signals supervise only the final answer. We propose PeRL-VL (Perception and Reasoning Learning for Vision-Language Models), a decoupled framework that separately improves visual perception and textual reasoning on top of RLVR. For perception, PeRL-VL introduces a VLM-based description reward that scores the model's self-generated image descriptions for faithfulness and sufficiency. For reasoning, PeRL-VL adds a text-only Reasoning SFT stage on logic-rich chain-of-thought data, enhancing coherence and logical consistency independently of vision. Across diverse multimodal benchmarks, PeRL-VL improves average Pass@1 accuracy from 63.3% (base Qwen2.5-VL-7B) to 68.8%, outperforming standard RLVR, text-only reasoning SFT, and naive multimodal distillation from GPT-4o.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
