arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.12869cs.RO
收藏

使用通过课程学习和Sim2Real增强的端到端深度强化学习穿越狭窄的差距

Flying Through a Narrow Gap Using End-to-end Deep Reinforcement Learning Augmented with Curriculum Learning and Sim2Real

Hong Kong Polytechnic University(香港理工大学);Purdue University(普渡大学);Northwestern Polytechnical University(西北工业大学);University of Hong Kong(香港大学)

Chenxi Xiao, Peng Lu, Qizhi He

摘要:

对于强化学习来说,穿越倾斜的狭窄间隙以前是一项棘手的任务,主要是由于两个挑战。首先,搜索可行的轨迹并不简单,因为差距背后的目标很难达到。其次,由于与间隙的窄尺寸相比相对较高的速度,Sim2Real之后的误差容限较低。由于碰撞损害的风险,收集真实世界数据的困难性使这一问题更加严重。在本文中,我们提出了一个端到端的强化学习框架,通过解决这两个问题成功地解决了这一任务。为了搜索动态可行的飞行轨迹,我们使用课程学习来引导智能体朝向障碍物后面的稀疏奖励。为了解决Sim2Real问题,我们提出了一个Sim2Real框架,可以将控制命令传输到真正的四旋翼机,而无需使用真实的飞行数据。据我们所知,我们的论文是第一个纯粹使用深度强化学习完成成功的间隙穿越任务的工作。

英文摘要:

Traversing through a tilted narrow gap is previously an intractable task for reinforcement learning mainly due to two challenges. First, searching feasible trajectories is not trivial because the goal behind the gap is difficult to reach. Second, the error tolerance after Sim2Real is low due to the relatively high speed in comparison to the gap's narrow dimensions. This problem is aggravated by the intractability of collecting real-world data due to the risk of collision damage. In this paper, we propose an end-to-end reinforcement learning framework that solves this task successfully by addressing both problems. To search for dynamically feasible flight trajectories, we use curriculum learning to guide the agent towards the sparse reward behind the obstacle. To tackle the Sim2Real problem, we propose a Sim2Real framework that can transfer control commands to a real quadrotor without using real flight data. To the best of our knowledge, our paper is the first work that accomplishes successful gap traversing task purely using deep reinforcement learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑