arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.15060cs.CVcs.RO
收藏

DriveGAN:迈向可控的高质量神经模拟

DriveGAN: Towards a Controllable High-Quality Neural Simulation

NVIDIA(英伟达);University of Toronto(多伦多大学);Vector Institute(向量研究所);MIT(麻省理工学院)

Seung Wook Kim, Jonah Philion, Antonio Torralba, Sanja Fidler

摘要:

逼真的模拟器对于训练和验证机器人系统至关重要。虽然大多数当代模拟器都是手工制作的,但构建模拟器的一种可扩展方式是使用机器学习来学习环境如何直接从数据中响应动作。在这项工作中,我们的目标是学习直接在像素空间中模拟动态环境,通过观察未注释的帧序列及其相关的动作对。我们介绍了一种新型的高质量神经模拟器,称为DriveGAN,它通过在没有监督的情况下解开不同的组件来实现可控性。除了转向控制之外,它还包括用于对场景的特征进行采样的控制,例如天气以及非玩家对象的位置。由于DriveGAN是一个完全可区分的模拟器,它还允许重新模拟给定的视频序列,提供代理再次驾驶通过记录的场景,可能采取不同的行动。我们在多个数据集上对DriveGAN进行训练,其中包括160小时的真实驾驶数据。我们展示了我们的方法大大超过了以前的数据驱动模拟器的性能,并允许以前没有探索过的新功能。

英文摘要:

Realistic simulators are critical for training and verifying robotics systems. While most of the contemporary simulators are hand-crafted, a scaleable way to build simulators is to use machine learning to learn how the environment behaves in response to an action, directly from data. In this work, we aim to learn to simulate a dynamic environment directly in pixel-space, by watching unannotated sequences of frames and their associated action pairs. We introduce a novel high-quality neural simulator referred to as DriveGAN that achieves controllability by disentangling different components without supervision. In addition to steering controls, it also includes controls for sampling features of a scene, such as the weather as well as the location of non-player objects. Since DriveGAN is a fully differentiable simulator, it further allows for re-simulation of a given video sequence, offering an agent to drive through a recorded scene again, possibly taking different actions. We train DriveGAN on multiple datasets, including 160 hours of real-world driving data. We showcase that our approach greatly surpasses the performance of previous data-driven simulators, and allows for new features not explored before.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑