arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.18490cs.ROcs.AI
收藏

基于课程学习的样本高效强化学习用于四旋翼机的鲁棒稳定控制

Curriculum-based Sample Efficient Reinforcement Learning for Robust Stabilization of a Quadrotor

Luleå University of Technology(吕勒奥理工大学)

Fausto Mauricio Lagos Suarez, Akshit Saradagi, Vidya Sumathy, Shruti Kotpaliwar, George Nikolakopoulos

AI总结:

本文提出一种样本高效课程学习方法,通过分阶段递增任务复杂度,提升四旋翼机在随机初始条件下的稳定控制性能,减少计算资源和训练时间。

摘要:

本文介绍了一种新颖的样本高效课程学习方法,用于训练端到端强化学习策略,以实现四旋翼机的鲁棒稳定控制。学习目标是通过直接控制电机RPM,从随机初始条件中同时稳定位置和偏航姿态,同时满足预设的暂态和稳态规范。该目标在空中检查应用中具有相关性,但传统的一阶段端到端强化学习需要大量计算资源和长时间训练。为解决这一挑战,本文受人类启发的课程学习启发,将学习目标分解为三个阶段的课程,逐步增加任务复杂度,并在阶段间转移知识。在所提出的课程中,策略依次学习悬停、平移与旋转自由度的耦合,以及对随机非零初始速度的鲁棒性,使用自定义奖励函数和回合截断条件。结果表明,与传统一阶段训练策略相比,所提课程学习方法在相同奖励函数和超参数下,显著减少了计算资源需求(样本)和收敛时间。课程训练的策略在模拟引擎(Gym-PyBullet-Drones)中,在随机初始条件和检查姿态跟踪场景中得到了彻底验证。一个展示我们结果的视频可在https://youtu.be/9wv6T4eezAU上观看。

英文摘要:

This article introduces a novel sample-efficient curriculum learning (CL) approach for training an end-to-end reinforcement learning (RL) policy for robust stabilization of a Quadrotor. The learning objective is to simultaneously stabilize position and yaw-orientation from random initial conditions through direct control over motor RPMs (end-to-end), while adhering to pre-specified transient and steady-state specifications. This objective, relevant in aerial inspection applications, is challenging for conventional one-stage end-to-end RL, which requires substantial computational resources and lengthy training times. To address this challenge, this article draws inspiration from human-inspired curriculum learning and decomposes the learning objective into a three-stage curriculum that incrementally increases task complexity, while transferring knowledge from one stage to the next. In the proposed curriculum, the policy sequentially learns hovering, the coupling between translational and rotational degrees of freedom, and robustness to random non-zero initial velocities, utilizing a custom reward function and episode truncation conditions. The results demonstrate that the proposed CL approach achieves superior performance compared to a policy trained conventionally in one stage, with the same reward function and hyperparameters, while significantly reducing computational resource needs (samples) and convergence time. The CL-trained policy's performance and robustness are thoroughly validated in a simulation engine (Gym-PyBullet-Drones), under random initial conditions, and in an inspection pose-tracking scenario. A video presenting our results is available at https://youtu.be/9wv6T4eezAU.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑