Recent large reasoning models (LRMs) driven by reinforcement learning algorithms (e.g., GRPO) have achieved remarkable performance on challenging reasoning tasks. However, these models suffer from overthinking, generating unnecessarily long and redundant reasoning even for simple questions, which substantially increases computational cost and response latency. While existing methods incorporate length rewards to GRPO to promote concise reasoning, they incur significant performance degradation. We identify the root cause: when rewards for correct but long rollouts are penalized, GRPO's group-relative advantage function can assign them negative advantages, actively discouraging valid reasoning. To overcome this, we propose Decoupled Reward Policy Optimization (DRPO), a novel framework that decouples the length-based learning signal of correct rollouts from incorrect ones. DRPO ensures that reward signals for correct rollouts are normalized solely within the positive group, shielding them from interference by negative samples. The DRPO's objective is grounded in integrating an optimized positive data distribution, which maximizes length-based rewards under a KL regularization, into a discriminative objective. We derive a closed-form solution for this distribution, enabling efficient computation of the objective and its gradients using only on-policy data and importance weighting. Of independent interest, this formulation is general and can incorporate other preference rewards of positive data beyond length. Experiments on mathematical reasoning tasks demonstrate DRPO's significant superiority over six efficient reasoning baselines. Notably, with a 1.5B model, our method achieves 77\% length reduction with only 1.1\% performance loss on simple questions like GSM8k dataset, while the follow-up baseline sacrifices 4.3\% for 68\% length reduction.
Department of Statistics（统计系）；LSE London, UK（伦敦大学学院）；Department of Mathematics（数学系）；Tsinghua University（清华大学）；School of Design LCC, UAL London, UK（伦敦艺术大学设计学院）；Department of Engineering Science（工程科学系）；University of Oxford（牛津大学）
GD$^2$PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization
Qwen Large Model Application Team, Alibaba（阿里巴巴通义千问大模型应用团队）；Renmin University of China（中国人民大学）；Peking University（北京大学）；ETH Zürich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；The Chinese University of Hong Kong（香港中文大学）
Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specifications
School of Computer Science, Georgia Institute of Technology, USA（美国佐治亚理工学院计算机科学学院）；Department of Computer Science and Engineering, Indian Institute of Technology Kanpur, India（印度理工学院坎浦尔分校计算机科学与工程系）
Think Dense, Not Long: Dynamic Decoupled Conditional Advantage for Efficient Reasoning
Beihang University（北航）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；National University of Defense Technology（国防科技大学）；Hong Kong Polytechnic University（香港理工大学）；Alibaba Group（阿里巴巴集团）
Each Reward to Its Relevant Tokens: Reward-Specific Token-Level Credit Assignment for Multi-Reward RL
University of Minnesota（明尼苏达大学）；National University of Singapore（新加坡国立大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Southern California（南加州大学）；University of Pennsylvania（宾夕法尼亚大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）
ConTraIRL: Factorized Contrastive Abstractions for Transferable IRL
School of Computing University of Georgia（乔治亚大学计算学院）；School of Computing Sciences & Computer Engineering The University of Southern Mississippi（密西西比大学计算科学与计算机工程学院）
DGRO: Enhancing LLM Reasoning via Exploration-Exploitation Control and Reward Variance Management
School of Mathematics and Statistics（数学与统计学学院）；Beijing Jiaotong University（北京交通大学）；Yau Mathematical Sciences Center（叶中科学中心）；Tsinghua University（清华大学）；Department of Mathematical Sciences（数学科学系）；Beijing, China（中国北京）；Zhongguancun Academy（中关村学院）；Yanqi Lake Beijing Institute of Mathematical Sciences and Applications（燕琪湖数学科学与应用研究所）；Academy of Mathematics and Systems Science（数学与系统科学研究院）；Chinese Academy of Sciences（中国科学院）；Microsoft Research AI4Science（微软研究院AI4Science）
Reward-Decomposed Reinforcement Learning for Immersive Video Role-Playing
Nanjing University（南京大学）；Shanghai Jiao Tong University（上海交通大学）；University of California, San Diego（加州大学圣地亚哥分校）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；School of Information Engineering, Beijing Institute of Graphic Communication（北京印刷学院信息工程学院）；Ant International, Ant Group（蚂蚁集团国际部）；Independent Researcher（独立研究者）
PAPO: Stabilizing Rubric Integration Training via Decoupled Advantage Normalization
University of Science and Technology of China（中国科学技术大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；University of Oxford（牛津大学）；Wuhan University（武汉大学）
