arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.03701cs.LGcs.ROstat.ML
收藏

通过仿真进行泛化:将模拟和真实数据集成到基于视觉的自主飞行的深度强化学习中

Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flight

Berkeley AI Research (BAIR), University of California, Berkeley(伯克利人工智能研究中心(BAIR),加州大学伯克利分校)

Katie Kang, Suneel Belkhale, Gregory Kahn, Pieter Abbeel, Sergey Levine

摘要:

深度强化学习为基于视觉的机器人控制提供了一种很有前途的方法。然而,这些模型的推广在很大程度上取决于可用于训练的数据的数量和种类。对于某些类型的机器人系统,例如脆弱的小型四旋翼机,这些数据可能难以获得。模拟渲染和物理可以提供更大的数据集,但这些数据本身质量较低:许多使现实世界的自主飞行问题具有挑战性的现象,如复杂的物理和气流,建模很差或根本没有,并且模拟和现实世界之间的系统差异通常是不可能消除的。在这项工作中,我们研究了如何将模拟和现实世界的数据结合到混合深度强化学习算法中。我们的方法使用真实世界的数据来了解系统的动态,并使用模拟数据来学习一个可推广的感知系统,该系统可以使机器人仅使用单目摄像机来避免碰撞。我们在一个真实世界的纳米飞行器碰撞避免任务上展示了我们的方法,表明只有一个小时的真实数据,四旋翼可以在各种照明条件和几何形状的新环境中避免碰撞。代码、建造飞行器的说明和实验视频可以在github.com/gkahn13/GtS上找到。

英文摘要:

Deep reinforcement learning provides a promising approach for vision-based control of real-world robots. However, the generalization of such models depends critically on the quantity and variety of data available for training. This data can be difficult to obtain for some types of robotic systems, such as fragile, small-scale quadrotors. Simulated rendering and physics can provide for much larger datasets, but such data is inherently of lower quality: many of the phenomena that make the real-world autonomous flight problem challenging, such as complex physics and air currents, are modeled poorly or not at all, and the systematic differences between simulation and the real world are typically impossible to eliminate. In this work, we investigate how data from both simulation and the real world can be combined in a hybrid deep reinforcement learning algorithm. Our method uses real-world data to learn about the dynamics of the system, and simulated data to learn a generalizable perception system that can enable the robot to avoid collisions using only a monocular camera. We demonstrate our approach on a real-world nano aerial vehicle collision avoidance task, showing that with only an hour of real-world data, the quadrotor can avoid collisions in new environments with various lighting conditions and geometry. Code, instructions for building the aerial vehicles, and videos of the experiments can be found at github.com/gkahn13/GtS

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑