Understanding and evaluating multi-step reasoning in LLMs at the level of individual steps remains a key challenge. Process reward models (PRMs) provide a solution by scoring each step, enabling fine-grained supervision and improved reliability. However, training them requires costly human annotation or computationally intensive rollout-based labeling. To solve this, we introduce MCNIG, a scalable method for automatically labeling the quality of individual reasoning steps in any verifiable domain. Its step score, net information gain (NetIG), improves upon single-reference information gain (IG) by comparing the most-supported correct answer against the most-supported incorrect one, yielding a robust signal even for long and structured outputs like code and SQL, where IG fails. We show that the signal produced by MCNIG correlates with human judgments of step quality, and we apply MCNIG labels to train PRMs that achieve the best average best-of-K accuracy across eight benchmarks spanning mathematics, code generation, text-to-SQL, and scientific QA. Crucially, MCNIG generates no rollouts, cutting labeling complexity to O(N) and making it up to X times cheaper than rollout-based methods at comparable label quality, which makes large-scale process supervision practical.
TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs
University of Chinese Academy of Sciences(中国科学院大学);Meituan(美团);MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室与智能信息处理重点实验室)
Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
Mila - Québec AI Institute, Canada(魁北克人工智能研究所);Huawei Noah's Ark Lab (Montreal Research Center), Canada(华为诺亚实验室(蒙特利尔研究中心));Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)
ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward
LMU Munich(慕尼黑大学);Harvard University(哈佛大学);University of Cambridge(剑桥大学);Mina AI;Konrad Zuse School of Excellence in Reliable AI (relAI)(康拉德·楚泽可靠人工智能卓越学校(relAI))
Auditing Data Membership in Reinforcement Learning With Verifiable Rewards
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Shanghai Jiao Tong University(上海交通大学);Shandong University(山东大学);Xidian University(西安电子科技大学);Wuhan University(武汉大学)
SkillScaffold: Scaffolding Agentic Reinforcement Learning with Policy-Adaptive Skill Credit Assignment and Selective Injection
Zhejiang University(浙江大学);Kuaishou Technology(快手科技);University of the Chinese Academy of Sciences(中国科学院大学);Zhejiang University of Science and Technology(浙江科技学院)
CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR
Qwen Large Model Application Team, Alibaba Institute of Automation, Chinese Academy of Sciences(文勤大模型应用团队,阿里巴巴自动化研究所,中国科学院);Qwen Large Model Application Team, Alibaba(文勤大模型应用团队,阿里巴巴)
Multilingual Verifier Bias in RLVR: Benchmark, Rollout Diagnosis, and the Cross-Lingual Selection Bottleneck
School of Engineering, Institute of Science Tokyo(科学东京大学工程学院);College of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院);Department of Electrical and Computer Engineering, National University of Singapore(新加坡国立大学电气与计算机工程学院)
Recycling Failures: Salvaging Exploration in RLVR via Fine-Grained Off-Policy Guidance
School of Artificial Intelligence, Beihang University, Beijing, China(北京航空航天大学人工智能学院);Hangzhou International Innovation Institute, Beihang University, Hangzhou, China(北京航空航天大学杭州国际创新研究院);University of Leicester, Leicester, United Kingdom(莱斯特大学);Nanyang Technological University, Singapore(南洋理工大学);The Hong Kong Polytechnic University, Hong Kong SAR, China(香港理工大学)
DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
Stanford University(斯坦福大学);University of Tokyo(东京大学);RIKEN AIP(日本理化学研究所革新智能研究中心);UC Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);Amazon AWS(亚马逊云科技)