arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.08152cs.RO
收藏

多保真强化学习用于时间最优四旋翼重规划

Multi-Fidelity Reinforcement Learning for Time-Optimal Quadrotor Re-planning

Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, USA(信息与决策系统实验室,麻省理工学院,剑桥,马萨诸塞州,美国)

Gilhyun Ryou, Geoffrey Wang, Sertac Karaman

AI总结:

针对无人机高速在线轨迹规划难题,提出多保真强化学习方法,协同训练规划策略与奖励估计器,利用多保真贝叶斯优化构建高保真模型,并在仿真与真实实验中验证,实现2毫秒级重规划,优于基线方法。

摘要:

无人机的高速在线轨迹规划由于需要对复杂动力学进行精确建模,同时又受计算能力限制,因此构成了一项重大挑战。本文提出了一种多保真强化学习方法(MFRL),旨在有效构建真实动力学模型,同时训练一种可随时部署于实时应用的规划策略。所提方法涉及规划策略与奖励估计器的协同训练;后者预测策略输出的性能,并通过多保真贝叶斯优化进行高效训练。该优化方法对不同保真度之间的相关性进行建模,从而在低保真基础上构建高保真模型,使得能够在有限的高保真实验下精确开发奖励模型。该框架进一步扩展,将真实世界飞行实验纳入强化学习训练中,使奖励模型能够精确反映真实世界约束,并拓宽策略在真实场景中的适用性。我们通过在仿真和真实环境中训练和测试规划策略,进行了严格的评估。与基线最小加加速度方法相比,训练得到的策略不仅能生成更快、更可靠的轨迹,而且平均能在2毫秒内完成轨迹更新,而基线方法则需要数分钟。

英文摘要:

High-speed online trajectory planning for UAVs poses a significant challenge due to the need for precise modeling of complex dynamics while also being constrained by computational limitations. This paper presents a multi-fidelity reinforcement learning method (MFRL) that aims to effectively create a realistic dynamics model and simultaneously train a planning policy that can be readily deployed in real-time applications. The proposed method involves the co-training of a planning policy and a reward estimator; the latter predicts the performance of the policy's output and is trained efficiently through multi-fidelity Bayesian optimization. This optimization approach models the correlation between different fidelity levels, thereby constructing a high-fidelity model based on a low-fidelity foundation, which enables the accurate development of the reward model with limited high-fidelity experiments. The framework is further extended to include real-world flight experiments in reinforcement learning training, allowing the reward model to precisely reflect real-world constraints and broadening the policy's applicability to real-world scenarios. We present rigorous evaluations by training and testing the planning policy in both simulated and real-world environments. The resulting trained policy not only generates faster and more reliable trajectories compared to the baseline snap minimization method, but it also achieves trajectory updates in 2 ms on average, while the baseline method takes several minutes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑