Reinforcement learning with verifiable rewards (RLVR) has driven substantial gains in language-model reasoning. However, models fine-tuned with RLVR need not remain compatible with weaker partners: gains achieved alone may erode under collaboration, while policy drift may place reasoning beyond a weaker model's predictive reach. To address both problems, we introduce Tandem RLVR (TRLVR), bringing the recently proposed tandem-training paradigm beyond proof-of-concept settings and into modern RLVR. In TRLVR, a senior learns from rewarded trajectories co-generated with a frozen junior, while the standard GRPO objective is applied only to senior-emitted tokens. Training Qwen3-4B-Instruct on competition math, TRLVR performs on par with a matched GRPO senior when reasoning alone. We characterize compatibility through the complementary lenses of handoff robustness and junior legibility. Under reasoning-step handoffs, the TRLVR senior retains nearly all of its solo performance with the junior, reducing the communication tax to a negligible loss. The senior's reasoning is also more legible to the junior: its token distribution stays more closely anchored to the junior's, and the junior predicts the senior's chain-of-thought more readily token by token. An ablation that explicitly regularizes GRPO toward the junior does not recover the same combination of capability and compatibility. These results provide initial evidence that tandem rollout structure is a promising direction for keeping RLVR reasoning gains accessible to weaker partners.
Promoting Efficient Reasoning with Verifiable Stepwise Reward
Meituan(美团);Fudan University(复旦大学);Beijing Institute of Technology(北京理工大学);University of Science and Technology of China(中国科学技术大学);East China Normal University(华东师范大学)
Contextual Rollout Bandits for Reinforcement Learning with Verifiable Rewards
School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Huawei(华为)
Instructing LLMs to Negotiate using Reinforcement Learning with Verifiable Rewards
Purdue University(普渡大学);California Institute of Technology(加州理工学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Massachusetts Institute of Technology(麻省理工学院)
Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards
Shanghai Innovation Institute, Shanghai, China(上海创新研究院);College of Future Information Technology, Fudan University, Shanghai, China(未来信息科技学院,复旦大学);Shanghai AI Laboratory, Shanghai, China(上海人工智能实验室);The Chinese University of Hong Kong, Hong Kong, China(香港中文大学)
Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China(中国科学院软件研究所信息处理实验室);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学);Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China(中国科学院大学网络安全学院);National Computer Network Emergency Response Technical Team/Coordination Center of China, Beijing, China(中国国家计算机网络应急技术配合中心)
Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards
Stanford University(斯坦福大学);UC Berkeley(加州大学伯克利分校);The University of Tokyo(东京大学);RIKEN AIP(理化学研究所AIP);Waseda University(早稻田大学);Georgia Tech(佐治亚理工学院);Northwestern University(西北大学);UCLA(加州大学洛杉矶分校);UNC Chapel Hill(北卡罗来纳大学教堂山分校);Yale University(耶鲁大学);University of Waterloo(滑铁卢大学);Independent Researcher(独立研究者);CUHK(香港中文大学);UT Southwestern Medical Center(西南医学中心);National University of Singapore(新加坡国立大学);UIUC(伊利诺伊大学厄巴纳-香槟分校);Amazon AWS AI(亚马逊AWS人工智能)
STRIDE: Strategic Trajectory Reasoning via Discriminative Estimation for Verifiable Reinforcement Learning
Kean University(基恩大学);Case Western Reserve University(凯斯西储大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);The Ohio State University(俄亥俄州立大学);Tongji University(同济大学);Duke Kunshan University(昆山杜克大学);Royal Melbourne Institute of Technology(皇家墨尔本理工大学)
Auditing Data Membership in Reinforcement Learning With Verifiable Rewards
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Shanghai Jiao Tong University(上海交通大学);Shandong University(山东大学);Xidian University(西安电子科技大学);Wuhan University(武汉大学)
The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward
Fudan University(复旦大学);INFLY TECH(INFLY科技);Griffith University(格里菲斯大学);University of Waterloo(滑铁卢大学);Shanghai Innovation Institute(上海创新研究院);Shanghai Academy of Artificial Intelligence for Science(上海人工智能科学研究院)