Reinforcement learning with verifiable rewards (RLVR) is central to training modern reasoning models, but the undisclosed training data raises concerns about benchmark contamination. Unlike pretraining methods, which optimize models using token-level probabilities, RLVR fine-tunes models based on reward feedback from self-generated reasoning trajectories, making conventional likelihood-based detection methods less effective. We show that RLVR induces a distinctive behavioral signature: prompts encountered during RLVR training result in more rigid and similar generations, while unseen prompts retain greater diversity. We introduce Min-$k$NN Distance, a simple black-box detector that quantifies this collapse by sampling multiple completions for a given prompt and computing the average of the $k$ smallest nearest-neighbor edit distances. Min-$k$NN Distance requires no access to the reference model or token probabilities. Experiments across multiple RLVR-trained reasoning models show that Min-$k$NN Distance reliably distinguishes RL-seen examples from unseen ones and outperforms existing membership inference and RL contamination detection baselines.
Recycling Failures: Salvaging Exploration in RLVR via Fine-Grained Off-Policy Guidance
School of Artificial Intelligence, Beihang University, Beijing, China(北京航空航天大学人工智能学院);Hangzhou International Innovation Institute, Beihang University, Hangzhou, China(北京航空航天大学杭州国际创新研究院);University of Leicester, Leicester, United Kingdom(莱斯特大学);Nanyang Technological University, Singapore(南洋理工大学);The Hong Kong Polytechnic University, Hong Kong SAR, China(香港理工大学)
Sparsity Curse: Understanding RLVR Model Parameter Space from Model Merging
Zhejiang University(浙江大学);Simon Fraser University(西蒙菲莎大学);The Chinese University of Hong Kong(香港中文大学);Zhejiang Key Lab of Accessible Perception and Intelligent Systems(浙江省可感知智能系统重点实验室)
Revisiting Entropy in Reinforcement Learning for Large Reasoning Models
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China(天津大学计算机科学与技术学院 TJUNLP 实验室,中国);Independent Researcher(独立研究者);College of Computer and Information Engineering, Tianjin Normal University, China(天津师范大学计算机与信息工程学院,中国)
The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models
College of Engineering and Computer Science, VinUniversity(Vin大学工程与计算机科学学院);Center for Environmental Intelligence, VinUniversity(Vin大学环境智能中心);University of Stuttgart(斯图加特大学);University of Notre Dame(诺特难大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Max Planck Research School for Intelligent Systems (IMPRS-IS)(马克斯·普朗克智能系统研究学校)
DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search
Stanford University(斯坦福大学);University of Tokyo(东京大学);RIKEN AIP(日本理化学研究所革新智能研究中心);UC Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);Amazon AWS(亚马逊云科技)
Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR
Jinesis Lab, University of Toronto & Vector Institute(Jinesis实验室,多伦多大学与向量研究所);Carnegie Mellon University(卡内基梅隆大学);Princeton University(普林斯顿大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);ELLIS Institute Tübingen(ELLIS研究所图宾根);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);EuroSafeAI
RLPR: Extrapolating RLVR to General Domains without Verifiers
Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Shanghai Qi Zhi Institute(上海启智研究院);Harbin Institute of Technology(哈尔滨工业大学);Beijing University of Posts and Telecommunications(北京邮电大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
STRIDE: Strategic Trajectory Reasoning via Discriminative Estimation for Verifiable Reinforcement Learning
Kean University(基恩大学);Case Western Reserve University(凯斯西储大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);The Ohio State University(俄亥俄州立大学);Tongji University(同济大学);Duke Kunshan University(昆山杜克大学);Royal Melbourne Institute of Technology(皇家墨尔本理工大学)
Scheduling Your LLM Reinforcement Learning with Reasoning Trees
Cranberry-Lemon University(Cranberry-Lemon 大学);University of the Witwatersrand(独立研究者);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR
New York University(纽约大学);University of Chicago(芝加哥大学);University of Waterloo(滑铁卢大学);University of Alberta(阿尔伯塔大学);NYU Shanghai(纽约大学上海分校);Alberta Machine Intelligence Institute (Amii)(阿尔伯塔机器智能研究所)
Spurious Rewards: Rethinking Training Signals in RLVR
University of Washington, Seattle, WA, USA(华盛顿大学);Allen Institute for Artificial Intelligence, Seattle, WA, USA(人工智能研究院);University of California, Berkeley, Berkeley, CA, USA(加州大学伯克利分校)
Text Before Vision: Staged Knowledge Injection Matters for Agentic RLVR in Ultra-High-Resolution Remote Sensing Understanding
National University of Defense Technology, China(国防科技大学);Beijing University of Posts and Telecommunications, China(北京邮电大学);University of the Chinese Academy of Sciences, China(中国科学院大学);Sichuan University, China(四川大学);Wuhan University, China(武汉大学);Chinese Academy of Science, China(中国科学院);Tsinghua University, China(清华大学);Shanghai Artificial Intelligence Laboratory, China(上海人工智能实验室);Renmin University of China, China(中国人民大学)
IRDS: Interpretable RLVR Data Selection via Verifier-Coupled Sparse Autoencoder Coverage
The Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州));Nanjing University of Aeronautics and Astronautics(南京航空航天大学);The 63rd Research Institute, National University of Defense Technology, Nanjing(国防科技大学第六三研究所,南京)
Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards
Shanghai Innovation Institute, Shanghai, China(上海创新研究院);College of Future Information Technology, Fudan University, Shanghai, China(未来信息科技学院,复旦大学);Shanghai AI Laboratory, Shanghai, China(上海人工智能实验室);The Chinese University of Hong Kong, Hong Kong, China(香港中文大学)
Generalization of RLVR Using Causal Reasoning as a Testbed
Johns Hopkins University(约翰霍普金斯大学);University of Maryland, College Park(马里兰大学学院公园分校);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft Research Asia(微软亚洲研究院);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)
CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR
Qwen Large Model Application Team, Alibaba Institute of Automation, Chinese Academy of Sciences(文勤大模型应用团队,阿里巴巴自动化研究所,中国科学院);Qwen Large Model Application Team, Alibaba(文勤大模型应用团队,阿里巴巴)
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Microsoft Research(微软研究院);University of Michigan(密歇根大学);Shanghai Jiao Tong University(上海交通大学);CUHKSZ(香港中文大学(深圳));THU(清华大学)