Group-Relative Policy Optimization (GRPO) has emerged as the standard for training reasoning capabilities in large language models through reinforcement learning. By estimating advantages using group-mean rewards rather than a learned critic, GRPO has enabled efficient scaling of reinforcement learning from verifiable rewards (RLVR). However, we identify a fundamental limitation: GRPO's mean baseline can assign positive advantages to incorrect solutions simply because they outperform a poorly-performing group average. It leads to overestimation of advantages and reinforcement of incorrect behaviours. To address this, we propose Correctness-Relative Policy Optimization (CoRPO), a simple modification to the GRPO objective that clips the minimum baseline to a fixed correctness threshold. We show that baseline clipping introduces a protective bias to advantage estimation that mitigates overfitting while preserving effective exploration. Empirically, CoRPO-trained models improve cross-domain reasoning, generalizing more consistently to out-of-domain (OOD) tasks. When trained on coding tasks, CoRPO outperforms GRPO on math, and vice-versa, indicating that CoRPO learns robust, transferable reasoning patterns rather than task-specific solutions.
CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO
School of Software and Microelectronics, Peking University, Beijing（北京大学软件与微电子学院）；School of Artificial Intelligence, Peking University, Beijing（北京大学人工智能学院）；School of Computer Science, Peking University, Beijing（北京大学计算机科学学院）；School of Future Technology, Peking University, Beijing（北京大学未来技术学院）
DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning
Morgan Stanley（摩根士丹利）；Clemson University（克莱姆森大学）；Arizona State University（亚利桑那州立大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；University of Notre Dame（圣母大学）；University of Arizona（亚利桑那大学）
Beijing University of Posts and Telecommunications（北京邮电大学）；School of Artificial Intelligence and Data Science, University of International Business and Economics（国际经济贸易大学人工智能与数据科学学院）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学北京校区人工智能学院）
Posterior-GRPO: Rewarding Reasoning Processes in Code Generation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）
Train Long, Think Short: Curriculum Learning for Efficient Reasoning
King Abdullah University of Science and Technology (KAUST), Saudi Arabia（卡布尔大学科学与技术大学）；Massachusetts Institute of Technology (MIT), Cambridge, MA, USA（麻省理工学院）；Princeton University, Princeton, NJ, USA（普林斯顿大学）
Hidden States Know Where Reasoning Diverges: Credit Assignment via Span-Level Wasserstein Distance
Beijing University of Posts and Telecommunications（北京邮电大学）；Meituan（美团）；Tianjin University（天津大学）；Renmin University of China（中国人民大学）；The University of Melbourne（墨尔本大学）；University of International Business and Economics（国际经贸大学）
Acting Less is Reasoning More! Teaching Model to Act Efficiently
The Chinese University of Hong Kong（香港中文大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴娜-香槟分校）；Princeton University（普林斯顿大学）；Sun Yat-sen University（中山大学）；Hong Kong University of Science and Technology（香港科学理工大学）
KTAE: A Model-Free Algorithm to Key-Tokens Advantage Estimation in Mathematical Reasoning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
R2IF: Aligning Reasoning with Decisions via Composite Rewards for Interpretable LLM Function Calling
Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, Shanghai, China（上海可信计算实验室，华东师范大学，上海，中国）；Huazhong University of Science and Technology（华中科技大学）；Nanyang Technological University（南洋理工大学）
A self-evolving multi-role collaborative framework with fine-grained difficulty guidance for innovative mathematical problem generation
School of Physics and Information Technology, Shaanxi Normal University（陕西师范大学物理与信息技术学院）；Department of Computing Technologies, Swinburne University of Technology（斯威本科技大学计算技术系）；Department of Software Engineering, University of Szeged（塞格德大学软件工程系）
