潜在想象促进自动驾驶赛车中的Zero-Shot转移
Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing
摘要：
世界模型在潜在的想象空间中学习行为，以提高深度强化学习（RL）算法的样本效率。在学习用于高维观测的世界模型时（例如，像素输入）已经在标准RL基准测试和一些游戏上变得可行，但它们在现实世界机器人应用中的有效性尚未被探索。在本文中，我们研究了这些代理如何推广到现实世界的自动驾驶汽车控制任务，其中先进的无模型深度RL算法失败了。特别是，我们为配备了高维激光雷达传感器的F1TENTH赛车机器人在一组测试轨道上设置了一系列时间圈任务，其复杂性逐渐增加。在这种连续控制设置中，我们表明，基于模型的代理能够在想象中学习大大优于无模型代理的性能，样本效率，成功完成任务，和泛化。此外，我们表明，基于模型的代理的泛化能力强烈依赖于他们的观察模型的选择。我们提供了广泛的经验证据的有效性世界模型提供了足够长的记忆视野在sim2real任务。
英文摘要：
World models learn behaviors in a latent imagination space to enhance the sample-efficiency of deep reinforcement learning (RL) algorithms. While learning world models for high-dimensional observations (e.g., pixel inputs) has become practicable on standard RL benchmarks and some games, their effectiveness in real-world robotics applications has not been explored. In this paper, we investigate how such agents generalize to real-world autonomous vehicle control tasks, where advanced model-free deep RL algorithms fail. In particular, we set up a series of time-lap tasks for an F1TENTH racing robot, equipped with a high-dimensional LiDAR sensor, on a set of test tracks with a gradual increase in their complexity. In this continuous-control setting, we show that model-based agents capable of learning in imagination substantially outperform model-free agents with respect to performance, sample efficiency, successful task completion, and generalization. Moreover, we show that the generalization ability of model-based agents strongly depends on the choice of their observation model. We provide extensive empirical evidence for the effectiveness of world models provided with long enough memory horizons in sim2real tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
