arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.04799cs.ROcs.AIcs.LG
收藏

RL-RRT:通过从RL策略学习可达性估计器的Kinodynamic运动规划

RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators from RL Policies

Google AI(谷歌人工智能);University of New Mexico(新墨西哥大学)

Hao-Tien Lewis Chiang, Jasmine Hsu, Marek Fiser, Lydia Tapia, Aleksandra Faust

摘要:

本文解决了基于采样的Kinodynamic运动规划面临的两个挑战:一种方法来确定良好的候选状态的局部过渡和随后的计算棘手的转向这些候选状态之间。通过结合基于采样的规划,快速探索随机树(RRT)和一个有效的Kinodynamic运动规划器通过机器学习,我们提出了一个有效的解决方案,长期规划Kinodynamic运动规划。首先,我们使用深度强化学习来学习避障策略,该策略将机器人的传感器观察映射到动作,该动作在规划期间用作本地规划器,在执行期间用作控制器。其次,我们训练可达性估计在监督的方式,预测RL政策的时间,以达到一个状态的存在障碍。最后,我们引入RL-RRT,它使用RL策略作为局部规划器,并将可达性估计器作为距离函数,以使树的生长偏向有希望的区域。我们评估我们的方法对三个kinodynamic系统,包括物理机器人实验。在所有三个机器人测试的结果表明,RL-RRT优于最先进的kinodynamic规划的效率,也提供了一个更短的路径完成时间比转向功能免费的方法。学习的本地规划器策略和伴随的可达性估计器展示了到先前看不见的实验环境的可转移性,使得RL-RRT快速,因为昂贵的计算被简单的神经网络推理取代。视频:https://youtu.be/dDMVMTOI8KY

英文摘要:

This paper addresses two challenges facing sampling-based kinodynamic motion planning: a way to identify good candidate states for local transitions and the subsequent computationally intractable steering between these candidate states. Through the combination of sampling-based planning, a Rapidly Exploring Randomized Tree (RRT) and an efficient kinodynamic motion planner through machine learning, we propose an efficient solution to long-range planning for kinodynamic motion planning. First, we use deep reinforcement learning to learn an obstacle-avoiding policy that maps a robot's sensor observations to actions, which is used as a local planner during planning and as a controller during execution. Second, we train a reachability estimator in a supervised manner, which predicts the RL policy's time to reach a state in the presence of obstacles. Lastly, we introduce RL-RRT that uses the RL policy as a local planner, and the reachability estimator as the distance function to bias tree-growth towards promising regions. We evaluate our method on three kinodynamic systems, including physical robot experiments. Results across all three robots tested indicate that RL-RRT outperforms state of the art kinodynamic planners in efficiency, and also provides a shorter path finish time than a steering function free method. The learned local planner policy and accompanying reachability estimator demonstrate transferability to the previously unseen experimental environments, making RL-RRT fast because the expensive computations are replaced with simple neural network inference. Video: https://youtu.be/dDMVMTOI8KY

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑