Large-scale verifiable prompts underpin the success of Reinforcement Learning with Verifiable Rewards (RLVR), but they contain many uninformative examples and are costly to expand further. Recent studies focus on better exploiting limited training data by prioritizing hard prompts whose rollout pass rate is 0. However, easy prompts with a pass rate of 1 also become increasingly prevalent as training progresses, thereby reducing the effective data size. To mitigate this, we propose Composition-RL, an offline data-augmentation approach that composes existing prompts before RL to address the pass-rate-1 bottleneck without modifying the RL optimizer. The composed dataset is generated and verified once from the fixed source pool and then used for RL training. Extensive experiments across model sizes from 4B to 30B show that Composition-RL consistently improves reasoning capability over RL trained on the original dataset. Performance can be further boosted with a curriculum variant of Composition-RL that gradually increases compositional depth over training. Additionally, Composition-RL enables more effective cross-domain RL by composing prompts drawn from different domains. All our code, data, and models are publicly available to facilitate future research.
Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalization
University of Chinese Academy of Sciences(中国科学院大学);Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所信息处理实验室);Qwen Team, Alibaba Group(阿里巴巴集团Qwen团队)
ComposerX: Multi-Agent Symbolic Music Composition with LLMs
University of Rochester(罗切斯特大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Hong Kong University of Science and Technology(香港科技大学);University of Surrey(萨里大学);The University of Manchester(曼彻斯特大学);Queen Mary University of London(伦敦玛丽女王大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
Department of Mechanical Engineering(机械工程系);Massachusetts Institute of Technology(麻省理工学院);Department of Civil and Environmental Engineering(土木与环境工程系);Schwarzman College of Computing(施瓦茨曼计算学院)
Verifiable Fine-Tuning for LLMs: Zero-Knowledge Training Proofs Bound to Data Provenance and Policy
1 Department of Computer Engineering, Istanbul Technical University (ITU), 34469 Istanbul, Turkey Email;2 Department of Computer Science, University of Malta, MSD 2080 Msida, Malta Email;3 Faculty of Electrical \& Computer Engineering, University of Prishtina ``Hasan Prishtina'', 10000 Prishtina, Kosovo Email;4 School of IT \& Engineering, ADA University, AZ1008 Baku, Azerbaijan Email;5 Faculty of Mathematics, Natural Sciences;Information Technologies (FAMNIT), University of Primorska, 6000 Koper, Slovenia Email;6 Faculty of Computer Science \& Engineering (FINKI), Ss.\ Cyril;Methodius University in Skopje, 1000 Skopje, North Macedonia Email