Reinforcement learning (RL) is a dominant paradigm for improving the reasoning abilities of large language models, yet its effectiveness varies across tasks and compute budgets. We propose a \emph{relative-budget} theory explaining this variation through a single quantity called relative budget $ξ:= H/\mathbb{E}[T]$, where $H$ is the generation horizon (token budget) and $T$ denotes the number of tokens until the first correct solution under a base policy. We show that $ξ$ determines sample efficiency by controlling reward variance and the likelihood of informative trajectories. Our analysis reveals three regimes: in the \emph{deficient} regime ($ξ\to 0$), informative trajectories are rare and the sample complexity explodes; in the \emph{balanced} regime ($ξ=Θ(1)$), informative trajectories occur with non-negligible probability and RL is maximally sample-efficient; and in the \emph{ample} regime ($ξ\to \infty$), learning remains stable but marginal gains per iteration diminish. We further provide finite-sample guarantees for online RL that characterize learning progress across these regimes. Specifically, in a case study under idealized distributional assumptions, we show that the relative budget grows linearly over iterations. Our empirical results confirm these predictions in realistic settings, identifying a budget $ξ\in [1.5, 2.0]$ that maximizes learning efficiency and coincides with peak reasoning performance.
BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
Department of Mechanical Engineering（机械工程系）；Massachusetts Institute of Technology（麻省理工学院）；Department of Civil and Environmental Engineering（土木与环境工程系）；Schwarzman College of Computing（施瓦茨曼计算学院）
RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains
University at Albany（阿尔巴尼大学）；Rutgers University（罗格斯大学）；Purdue University（普渡大学）；Independent Researcher（独立研究员）；Emory University（埃默里大学）；Georgia Institute of Technology（佐治亚理工学院）
Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China（中国科学院软件研究所信息处理实验室）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学）；Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China（中国科学院大学网络安全学院）；National Computer Network Emergency Response Technical Team/Coordination Center of China, Beijing, China（中国国家计算机网络应急技术配合中心）
From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement
Duke University（杜克大学）；Adobe Inc.（奥多比公司）；Oregon State University（俄勒冈州立大学）；Pennsylvania State University（宾夕法尼亚州立大学）；National University of Singapore（新加坡国立大学）；Amazon（亚马逊）
GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
Key Lab of HCST (PKU), MOE；SCS, Peking University, Beijing, China；Tencent Inc., Shenzheng, China；Hong Kong University of Science；Department of Computer Science, University of Texas at Dallas, Richardson, USA；School of Computing Science, Simon Fraser University, Burnaby, BC, Canada
Contextual Rollout Bandits for Reinforcement Learning with Verifiable Rewards
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；Huawei（华为）
BV-Blend: Uncertainty-Weighted Historical Baselines for Stable Critic-Free RL with Verifiable Rewards
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China（知识驱动人机智能工程研究中心，教育部，中国）；International Center of Future Science, Jilin University（未来科学国际中心，吉林大学）
