RuCL：基于分层评分标准的课程学习用于多模态大语言模型推理
RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning
AI总结：
RuCL通过分层评分标准课程学习提升多模态大语言模型的推理能力，实现7.83%的准确率提升。
摘要：
可验证奖励的强化学习（RLVR）已逐渐成为增强多模态大语言模型（MLLMs）推理能力的一种主流范式。然而，仅依赖结果监督会带来奖励黑客的风险，即模型会学习到虚假的推理模式以满足最终答案的检查。尽管最近的基于评分标准的方法提供了细粒度的监督信号，但它们却因实例级生成的高计算成本和将所有评分标准视为同等可学习而导致训练动态效率低下。在本文中，我们提出了基于分层的评分标准课程学习（RuCL），一种新的框架，通过将关注点从数据选择转移到奖励设计来重新制定课程学习。RuCL为广泛适用性生成通用评分标准，并根据模型的能力对其进行分层。通过在训练过程中动态调整评分标准的权重，RuCL引导模型从掌握基础感知到解决高级逻辑推理。在各种视觉推理基准上的广泛实验表明，RuCL在Qwen2.5-VL-7B模型上实现了平均+7.83%的显著提升，达到60.06%的最先进准确率。
英文摘要：
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a prevailing paradigm for enhancing reasoning in Multimodal Large Language Models (MLLMs). However, relying solely on outcome supervision risks reward hacking, where models learn spurious reasoning patterns to satisfy final answer checks. While recent rubric-based approaches offer fine-grained supervision signals, they suffer from high computational costs of instance-level generation and inefficient training dynamics caused by treating all rubrics as equally learnable. In this paper, we propose Stratified Rubric-based Curriculum Learning (RuCL), a novel framework that reformulates curriculum learning by shifting the focus from data selection to reward design. RuCL generates generalized rubrics for broad applicability and stratifies them based on the model's competence. By dynamically adjusting rubric weights during training, RuCL guides the model from mastering foundational perception to tackling advanced logical reasoning. Extensive experiments on various visual reasoning benchmarks show that RuCL yields a remarkable +7.83% average improvement over the Qwen2.5-VL-7B model, achieving a state-of-the-art accuracy of 60.06%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
