Although reinforcement learning (RL) improves the reasoning capabilities of large language models (LLMs), GRPO-style methods can suffer from a positive-credit contamination failure mode. Specifically, uniform credit assignment can reinforce locally erroneous tail tokens alongside useful tokens within successful trajectories. To address this problem, we propose Tail-Aware Credit calibratiOn (TACO), which calibrates the positive credit assigned to such tokens without token-level error labels. To this end, TACO lets the policy serve as its own reference, assessing each sampled token's rarity relative to the policy's local expectations to compute a tail-risk score. Using this score as a proxy for potential local errors, TACO softly reduces positive credit for high tail-risk tokens while retaining nonzero weights, limiting their reinforcement without eliminating opportunities to learn useful rare patterns. Experiments across four LLMs and eight reasoning benchmarks show that TACO consistently improves over GRPO-style baselines. Notably, TACO also improves training stability, supporting sustained performance gains in long-horizon RL. The source code is available at: https://github.com/xiuyilou/TACO.
Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling
South China University of Technology(华南理工大学);Tsinghua University(清华大学);The Hong Kong University of Science and Technology(香港科技大学);Huawei 2012 Laboratories(华为2012实验室);Shenzhen Technology University(深圳技术大学)
CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR
Qwen Large Model Application Team, Alibaba Institute of Automation, Chinese Academy of Sciences(文勤大模型应用团队,阿里巴巴自动化研究所,中国科学院);Qwen Large Model Application Team, Alibaba(文勤大模型应用团队,阿里巴巴)
Orchestrating Tokens and Sequences: Dynamic Hybrid Policy Optimization for RLVR
School of Informatics, Xiamen University(厦门大学信息学院);LLM Team, Shopee Pte. Ltd.(Shopee 股份有限公司语言模型团队);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院)
Hidden States Know Where Reasoning Diverges: Credit Assignment via Span-Level Wasserstein Distance
Beijing University of Posts and Telecommunications(北京邮电大学);Meituan(美团);Tianjin University(天津大学);Renmin University of China(中国人民大学);The University of Melbourne(墨尔本大学);University of International Business and Economics(国际经贸大学)
Mechanistically Interpreting the Role of Sample Difficulty in RLVR for LLMs
Beijing Jiaotong University(北京交通大学);AntGroup(蚂蚁集团);Northwestern Polytechnical University(西北工业大学);University of Leeds(利兹大学);University of Southampton(南安普顿大学)
Structured Role-Aware Policy Optimization for Multimodal Reasoning
School of Computing & Data Science, The University of Hong Kong(计算与数据科学学院,香港大学);School of Computing & Data Science and Institute of Data Science, The University of Hong Kong(计算与数据科学学院和数据科学研究所,香港大学)
STRIDE: Strategic Trajectory Reasoning via Discriminative Estimation for Verifiable Reinforcement Learning
Kean University(基恩大学);Case Western Reserve University(凯斯西储大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);The Ohio State University(俄亥俄州立大学);Tongji University(同济大学);Duke Kunshan University(昆山杜克大学);Royal Melbourne Institute of Technology(皇家墨尔本理工大学)
CoSPlay: Cooperative Self-Play at Test-Time with Self-Generated Code and Unit Test
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Institute of Deep Perception Technology, JITRI, Wuxi, China(深度感知技术研究院,无锡,中国)
Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所信息处理实验室);University of Chinese Academy of Sciences(中国科学院大学);Xiaohongshu Inc(小红书公司)
Improving Generalization Robustness of Multimodal RLVR
National University of Singapore(新加坡国立大学);DAMO Academy Alibaba Group(阿里巴巴达摩院);Hupan Lab(湖畔实验室);Zhejiang University(浙江大学);University of California Berkeley(加州大学伯克利分校);Rochester Institute of Technology(罗切斯特理工学院);Georgia Institute of Technology(佐治亚理工学院);Renmin University of China(中国人民大学);Hong Kong University of Science and Technology(香港科技大学)
Do Math Reasoning LLMs Help Predict the Impact of Public Transit Events?
Department of Civil Engineering and Engineering Mechanics, Columbia University(哥伦比亚大学土木工程与工程力学系);Engineering Mechanics, Columbia University(哥伦比亚大学工程力学系);Department of Computer Science, Columbia University(哥伦比亚大学计算机科学系);Data Science Institute, Columbia University(哥伦比亚大学数据科学研究院)
TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs
University of Chinese Academy of Sciences(中国科学院大学);Meituan(美团);MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室与智能信息处理重点实验室)
Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Zhejiang University(浙江大学)
Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents
College of Computer, National University of Defense Technology(国防科技大学计算机学院);Intelligent Game and Decision Lab (IGDL)(智能游戏与决策实验室);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院)
Beyond Timestamps: Decision-Aligned On-Policy Distillation for Long-Horizon Agents
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing(北京未来区块链与隐私计算高精尖创新中心);Tsinghua University(清华大学);Beihang University(北京航空航天大学)