重新思考视觉语言模型的强化学习扩展：一个透明的、从零开始的框架与综合评估方案
Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme
AI总结：
本文提出了一个透明且从零开始的视觉语言模型强化学习框架及标准化评估方案，实验发现RL在泛化上始终优于SFT，为该领域提供了可复现的基线。
摘要：
强化学习（RL）近期在提升大型语言模型的推理能力方面展现出巨大潜力，目前正被积极扩展至视觉语言模型（VLMs）。然而，现有RL在VLMs中的应用通常依赖重度工程的框架，这阻碍了可复现性与可访问性，同时缺乏标准化的评估协议，使得比较结果或解释训练动态变得困难。本工作引入了一个透明的、从零开始的VLMs强化学习框架，提供了一个最小但功能完备的四步流程，并在多个模型和数据集上进行了验证。此外，提出了一种标准化的评估方案，以评估训练动态和反思行为。在视觉推理任务上的大量实验揭示了关键实证发现：响应长度对随机种子敏感，反思与输出长度相关，并且即使使用高质量数据，RL在泛化能力上也始终优于监督微调（SFT）。这些发现连同所提出的框架，旨在建立一个可复现的基线，并支持更广泛地参与基于RL的VLM研究。
英文摘要：
Reinforcement learning (RL) has recently shown strong potential in improving the reasoning capabilities of large language models and is now being actively extended to vision-language models (VLMs). However, existing RL applications in VLMs often rely on heavily engineered frameworks that hinder reproducibility and accessibility, while lacking standardized evaluation protocols, making it difficult to compare results or interpret training dynamics. This work introduces a transparent, from-scratch framework for RL in VLMs, offering a minimal yet functional four-step pipeline validated across multiple models and datasets. In addition, a standardized evaluation scheme is proposed to assess training dynamics and reflective behaviors. Extensive experiments on visual reasoning tasks uncover key empirical findings: response length is sensitive to random seeds, reflection correlates with output length, and RL consistently outperforms supervised fine-tuning (SFT) in generalization, even with high-quality data. These findings, together with the proposed framework, aim to establish a reproducible baseline and support broader engagement in RL-based VLM research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
