Robots deployed in the real world rarely operate under a single fixed dynamics model: wind changes, payloads vary, batteries drain, contacts shift, and hardware wears. Yet most learning-based controllers are trained once and deployed as if learning were complete. This prevents the robot from using deployment experience to further improve task performance. In this work, we propose a continual learning framework that uses real-world experience to improve robot policies under hidden and recurring dynamics. Our method learns a condition-aware dynamics model from real state-action trajectories by combining an analytical physics prior with a neural residual for unmodeled effects. A recurrent encoder infers the current hidden condition from recent interaction, and this estimate conditions both the residual model and the policy. Policy learning is performed via differentiable simulation using diverse learned dynamics sampled from the latent model. At deployment, these sampled conditions are replaced by conditions inferred online from recent real interaction, allowing the policy to recover recurring dynamics by recognition rather than residual re-fitting. Through extensive simulation studies and real-world experiments, we demonstrate that the framework improves policy performance under diverse unobserved disturbances. On real quadrotor trajectory tracking under changing wind, the policy recovers from recurring disturbances in roughly 1s, about 5x faster than online residual re-fitting. It also reduces large-disturbance hover and tracking errors by 65.7% and 53.3% over the state-of-the-art online adaptation approaches
Learning from Mistakes: Rollout-Retrieval Lifelong Policy Learning for Autonomous Driving
School of Mechanical Engineering, Beijing Institute of Technology(北京理工大学机械工程学院);School of Mechanical and Aerospace Engineering, Nanyang Technological University(南洋理工大学机械与航空航天工程学院)
DTCCL: Disengagement-Triggered Contrastive Continual Learning for Autonomous Bus Planners
School of Vehicle and Mobility, Tsinghua University(车辆与移动系统学院,清华大学);Dongfeng Motor Corporation Research and Development General Institute(东风汽车集团研发总院);Dongfeng Yuexiang Technology Co., Ltd.(东风悦铃科技有限公司)
HyperDCM: Dynamic Cluster Memory Replay in Hyperbolic Space for Continual Robotic Navigation Across Scenes
Software Engineering Institute, East China Normal University(华东师范大学软件工程学院);School of Geospatial Information, Information Engineering University(信息工程大学地理空间信息学院);University of Shanghai for Science and Technology(上海理工大学);Shanghai Jiao Tong University(上海交通大学)
Extreme-RGMT: Continual Learning of Highly Dynamic Skills for Robust Generalist Humanoid Control
School of Automation, Beijing Institute of Technology(北京理工大学自动化学院);Humanoid Robotics (Shanghai) Co., Ltd.(人形机器人(上海)有限公司);School of Mechanical Engineering, Shandong University(山东大学机械工程学院)
Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Robotics
Technical University of Munich(慕尼黑工业大学);University of California, Berkeley(加州大学伯克利分校);Technische Universität Berlin(柏林工业大学);L3S Research Center, Leibniz Universität Hannover(莱布尼茨汉诺威大学L3S研究中心)
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)(计算机科学与技术学院,哈尔滨工业大学(深圳));Department of Computer Science and Technology, Ocean University of China(计算机科学与技术系,中国海洋大学);Huawei Noah’s Ark Lab(华为诺亚实验室)