This article introduces a novel sample-efficient curriculum learning (CL) approach for training an end-to-end reinforcement learning (RL) policy for robust stabilization of a Quadrotor. The learning objective is to simultaneously stabilize position and yaw-orientation from random initial conditions through direct control over motor RPMs (end-to-end), while adhering to pre-specified transient and steady-state specifications. This objective, relevant in aerial inspection applications, is challenging for conventional one-stage end-to-end RL, which requires substantial computational resources and lengthy training times. To address this challenge, this article draws inspiration from human-inspired curriculum learning and decomposes the learning objective into a three-stage curriculum that incrementally increases task complexity, while transferring knowledge from one stage to the next. In the proposed curriculum, the policy sequentially learns hovering, the coupling between translational and rotational degrees of freedom, and robustness to random non-zero initial velocities, utilizing a custom reward function and episode truncation conditions. The results demonstrate that the proposed CL approach achieves superior performance compared to a policy trained conventionally in one stage, with the same reward function and hyperparameters, while significantly reducing computational resource needs (samples) and convergence time. The CL-trained policy's performance and robustness are thoroughly validated in a simulation engine (Gym-PyBullet-Drones), under random initial conditions, and in an inspection pose-tracking scenario. A video presenting our results is available at https://youtu.be/9wv6T4eezAU.
Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles
School of Automation and Perception, Shanghai Jiao Tong University(自动化与感知学院,上海交通大学);Faculty of Aerospace Engineering, Delft University of Technology(航空航天工程学院,代尔夫特理工大学)
A Unidirectionally Connected FAS Approach for 6-DOF Quadrotor Control
Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology(广东省全驱动系统控制理论与技术重点实验室);Southern University of Science and Technology(南方科技大学);Center for Control Theory and Guidance Technology(控制理论与导航技术中心);Harbin Institute of Technology(哈尔滨工业大学)
Learning When to Act: Communication-Efficient Reinforcement Learning via Run-Time Assurance
Department of Mechanical Engineering, Iowa State University(爱荷华州立大学机械工程系);Department of Weapons, Robotics, and Control Engineering, United States Naval Academy(美国海军学院武器、机器人与控制工程系);Navy Center for Applied Research in Artificial Intelligence (NCARAI), U.S. Naval Research Laboratory(美国海军人工智能应用研究中心(NCARAI))