Group Relative Policy Optimization (GRPO) is an effective recipe for training reasoning models with verifier-based outcome rewards, but its supervision is sparse: when all sampled trajectories for a prompt receive the same verifier reward, the group-relative advantage collapses to zero and learning stalls. Outcome-only rewards also provide no step-level credit assignment, limiting exploration and making it harder to learn robust reasoning. We present VeriGate (Verifier-Gated Step-Level GRPO), a verifier-gated extension of GRPO that addresses these limitations with three design choices. First, VeriGate keeps the verifier in charge whenever verifier rewards induce a meaningful preference among sampled trajectories, and uses process supervision only when verifier rewards are degenerate. Second, instead of collapsing Process Reward Model (PRM) step scores into a single trajectory reward, VeriGate converts them into future-cumulated rewards to assign continuation-aware credit. Third, VeriGate transforms these rewards into group-normalized token-level advantages, restoring informative gradients and fine-grained credit assignment while remaining less susceptible to reward hacking than methods that optimize aggregated PRM scores. Empirically, training on MATH with 1.5B and 7B Qwen2.5-Instruct models and evaluating on six reasoning benchmarks, VeriGate improves average accuracy by about 20% and 12% for 1.5B and 7B models respectively, substantially reduces zero-gradient failures, decreases reward-hacking behavior, and improves reasoning quality relative to outcome-only GRPO and PRM-as-outcome baselines.
Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO
Southern University of Science and Technology(南方科技大学);National University of Singapore(新加坡国立大学);Harbin Institute of Technology, Weihai(哈尔滨工业大学(威海));Huawei Noah’s Ark Lab(华为诺亚方舟实验室);City University of Hong Kong(香港城市大学)
Multilingual Verifier Bias in RLVR: Benchmark, Rollout Diagnosis, and the Cross-Lingual Selection Bottleneck
School of Engineering, Institute of Science Tokyo(科学东京大学工程学院);College of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院);Department of Electrical and Computer Engineering, National University of Singapore(新加坡国立大学电气与计算机工程学院)
MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models
Université Paris-Saclay(巴黎萨克雷大学);ÉTS Montréal(蒙特利尔ÉTS);ILLS – International Laboratory on Learning Systems(学习系统国际实验室);Mila – Quebec AI Institute(魁北克人工智能研究所);Mistral AI;LIVIA;CNRS, CentraleSupélec(国家科学研究中心,中央超导大学)
Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling
South China University of Technology(华南理工大学);Tsinghua University(清华大学);The Hong Kong University of Science and Technology(香港科技大学);Huawei 2012 Laboratories(华为2012实验室);Shenzhen Technology University(深圳技术大学)
Train Long, Think Short: Curriculum Learning for Efficient Reasoning
King Abdullah University of Science and Technology (KAUST), Saudi Arabia(卡布尔大学科学与技术大学);Massachusetts Institute of Technology (MIT), Cambridge, MA, USA(麻省理工学院);Princeton University, Princeton, NJ, USA(普林斯顿大学)
AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
Rutgers University(新泽西州立大学);Nanyang Technological University(南洋理工大学);University of Connecticut(康涅狄格大学);Fudan University(复旦大学);NVIDIA Research(NVIDIA研究);Red Hat AI Innovation(红帽AI创新);MIT-IBM Watson AI Lab(MIT-IBM沃森AI实验室);Massachusetts Institute of Technology(麻省理工学院)