Reinforcement learning with verifiable rewards (RLVR) has shown great potential to enhance the reasoning ability of large language models (LLMs). However, due to the limited amount of information provided during the RLVR process, the model can only engage in largely blind exploration, which often results in failure on challenging problems. To provide additional information for the RLVR process without relying on a teacher model, we propose A$^2$D, an Adaptive Ability Decomposing method for enhancing the effectiveness of RLVR. Specifically, we first train a decomposer via RLVR without distillation, enabling it to decompose complex questions into a set of simpler sub-questions. Next, we use this decomposer to annotate sub-questions for each question in the training dataset, and then train the reasoner under RLVR with sub-question guidance. To better understand A$^2$D, we first compare its performance with competitive baselines, showing its effectiveness. Next, we observe that our method functions as a plug-and-play module that can be applied to different RLVR algorithms. Furthermore, we conduct an analysis of the decomposer, revealing how the RLVR process affects its performance and behavior, and which type of guidance is better suited for enhancing the reasoner's exploration and exploitation abilities.
Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models
The Hong Kong Polytechnic University(香港理工大学);Zhejiang University(浙江大学);University of Electronic Science and Technology of China(电子科技大学);Reallm Labs(Reallm 实验室);Amazon(亚马逊);The Hong Kong University of Science and Technology(香港理工大学)
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction
School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);NiuTrans Research(NiuTrans研究院);Institute of Psychology, CAS(中国科学院心理研究所);Kunming University of Science and Technology(昆明理工大学)
Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learning
National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家实验室,南京大学);School of Artificial Intelligence, Nanjing University(人工智能学院,南京大学);School of Data Science, The Chinese University of Hong Kong, Shenzhen(数据科学学院,香港中文大学(深圳));Department of Computer Science and Engineering, The Chinese University of Hong Kong(计算机科学与工程系,香港中文大学)
Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing
Unlocking Noisy Real-World Corpora for Foundation Model Pre-Training via Quality-Aware Tokenization
Broad Institute of MIT and Harvard, Cambridge, MA, USA(麻省理工学院与哈佛大学Broad研究所);Massachusetts Institute of Technology, Cambridge, MA, USA(麻省理工学院);Koch Institute for Integrative Cancer Research, MIT, Cambridge, MA, USA(麻省理工学院Koch整合癌症研究 institute);Department of Neurology and Neurosurgery, McGill University, Montreal, Canada(麦吉尔大学神经学与神经外科系);The Montreal Neurological Hospital-Institute, Montreal, Canada(蒙特利尔神经科学医院-研究所)
Unlocking Metasurface Practicality for B5G Networks: AI-assisted RIS Planning
NEC Laboratories Europe(NEC欧洲实验室);Université Paris-Saclay(巴黎萨克雷大学);CentraleSupélec(中央理工-高等电力学院);University Carlos III of Madrid(马德里卡洛斯三世大学);Flyhound Co.(Flyhound公司);i2cat Foundation(i2cat基金会);ICREA(加泰罗尼亚研究与高级研究院)
CALM Before the STORM: Unlocking Native Reasoning for Optimization Modeling
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Qwen Team, Alibaba Inc.(阿里巴巴集团文勤团队);Shanghai University of Finance and Economics(上海财经大学);Southern University of Science and Technology(南方科技大学);Shenzhen Loop Area Institute (SLAI)(深圳环湖研究院)
Efficient and Stable Reinforcement Learning for Diffusion Language Models
State Key Laboratory of Cognitive Intelligence, University of Science(认知智能国家重点实验室,科学大学);City University of Hong Kong, Hong Kong, China(香港城市大学,香港,中国);Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China(人工智能学院,中国人民大学,北京,中国);School of Software and Microelectronics, Peking University, Beijing, China(软件与微电子学院,北京大学,北京,中国)
Self-Compression of Chain-of-Thought via Multi-Agent Reinforcement Learning
Renmin University of China(中国人民大学);Xiaohongshu Inc.(小红书公司);Institute of Automation,Chinese Academy of Sciences(中国科学院自动化研究所);University of Southern California(南加州大学);Shandong University(山东大学)
WST: Weak-to-Strong Knowledge Transfer via Reinforcement Learning
Wharton AI & Analytics Initiative(沃顿人工智能与分析倡议);University of Pennsylvania(宾夕法尼亚大学);Department of Computer and Information Science(计算机与信息科学系);Department of Physics and Astronomy(物理学与天文学系)