Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training
University of Illinois Chicago(伊利诺伊大学香槟分校);Caltech(加州理工学院);RPI(罗切斯特理工学院);MBZUAI(澳门大学人工智能研究院);University of Chicago(芝加哥大学);NEC Laboratories America(NEC美国实验室)
Zhengyao Gu, Jonathan Light, Raul Astudillo, Ziyu Ye, Langzhou He, Henry Peng Zou, Wei Cheng, Santiago Paternain, Philip S. Yu, Yisong Yue
备注:
37 pages, 8 figures, 1 table. Preprint under review. Equal contribution by first two authors
Post-training large foundation models with reinforcement learning typically relies on massive and heterogeneous datasets, making effective curriculum learning both critical and challenging. In this work, we propose ACTOR-CURATOR, a scalable and fully automated curriculum learning framework for reinforcement learning post-training of large language models (LLMs). ACTOR-CURATOR learns a neural curator that dynamically selects training problems from large problem banks by directly optimizing for expected policy performance improvement. We formulate problem selection as a non-stationary stochastic bandit problem, derive a principled loss function based on online stochastic mirror descent, and establish regret guarantees under partial feedback. Empirically, ACTOR-CURATOR consistently outperforms uniform sampling and strong curriculum baselines across a wide range of challenging reasoning benchmarks, demonstrating improved training stability and efficiency. Notably, it achieves relative gains of 28.6% on AIME2024 and 30.5% on ARC-1D over the strongest baseline and up to 80% speedup. These results suggest that ACTOR-CURATOR is a powerful and practical approach for scalable LLM post-training.
Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics
University of Luebeck(吕贝克大学);KION Group AG(凯傲集团股份公司);Montanuniversität Leoben(莱奥本矿业大学)
我们提出了一种深度强化学习方法来解决仓库场景中的无地图导航问题。在我们的方法中,自动引导车辆配备了LiDAR和正面RGB传感器,并学习执行有针对性的导航任务。挑战在于用于学习的正样本的稀疏性、具有部分可观测性的多模态传感器感知、对精确转向机动的需求以及长训练周期。为了解决这些问题,我们提出NavACL-Q作为一种自动课程学习的方法,结合分布式版本的软演员-评论家算法。学习算法的性能在一个看不见的仓库环境中进行了详尽的评估,以验证学习策略的鲁棒性和通用性。NVIDIA Isaac Sim中的结果表明,经过训练的智能体的性能明显优于NVIDIA Isaac Sim提供的基于地图的导航管道,其智能体目标距离增加了3米,初始相对智能体目标旋转了45度。消融研究还表明,NavACL-Q极大地促进了学习过程,与随机开始的训练相比,性能增益约为40%,并且使用预训练的特征提取器明显提高了约60%的性能。
Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning
VU Amsterdam(阿姆斯特丹自由大学);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校);Leiden University(莱顿大学);University of Warsaw(华沙大学);Simon Fraser University(西蒙菲莎大学);The University of Hong Kong(香港大学)
Continuous World Coverage Path Planning for Fixed-Wing UAVs using Deep Reinforcement Learning
TUM School of Engineering and Design, Technical University of Munich(慕尼黑技术大学工程与设计学院);Dept. of Electrical Engineering and Computer Sciences, University of California, Berkeley(加州大学伯克利分校电气工程与计算机科学系)
Learning to Recover: Dynamic Reward Shaping with Wheel-Leg Coordination for Fallen Robots
Humanoids and Human-Centered Mechatronics (HHCM), Istituto Italiano di Tecnologia(人类oids与人本机电系统(HHCM),意大利技术研究所);Ph.D. Program of National Interest in Robotics and Intelligent Machines (DRIM), University of Genova(机器人与智能机器国家利益博士项目(DRIM),热那亚大学);Department of Computer Science, University College London(计算机科学系,伦敦大学学院)
Dmsh: A Multi-Agent Reinforcement Learning Framework for All-Quad Mesh Generation
Department of Mechanical Engineering, Indian Institute of Technology Madras(印度理工学院马德拉斯分校机械工程系);Institute of Continuum Mechanics, Leibniz Universität Hannover(莱比锡大学汉诺威连续力学研究所);Institute of Structural Mechanics, Bauhaus-Universität Weimar(魏玛包豪斯大学结构力学研究所);Department of Civil and Systems Engineering, Johns Hopkins University(约翰霍普金斯大学土木与系统工程系)