使用潜在空间生成世界模型减轻自动驾驶模仿学习中的协变量转移
Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models
AI总结：
提出用潜在空间生成世界模型解决自动驾驶协变量转移问题，训练时利用世界模型减轻该问题，无需大量训练数据，还引入新感知编码器，实验显示相比之前有显著改进。
摘要：
我们建议使用潜在空间生成世界模型来解决自动驾驶中的协变量转移问题。世界模型是一种神经网络，能够根据过去的状态和动作预测智能体的下一个状态。通过在训练期间利用世界模型，驾驶策略有效地减轻了协变量转移，而无需大量的训练数据。在端到端训练期间，我们的策略学习如何通过与人类示范中观察到的状态对齐来从错误中恢复，以便在运行时它可以从训练分布之外的扰动中恢复。此外，我们引入了一种新颖的基于Transformer的感知编码器，该编码器采用多视图交叉注意力和学习到的场景查询。我们展示了定性和定量结果，证明在CARLA模拟器中的闭环测试中比现有技术有显著改进，以及在CARLA和NVIDIA的DRIVE Sim中处理扰动的能力。
英文摘要：
We propose the use of latent space generative world models to address the covariate shift problem in autonomous driving. A world model is a neural network capable of predicting an agent's next state given past states and actions. By leveraging a world model during training, the driving policy effectively mitigates covariate shift without requiring an excessive amount of training data. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations, so that at runtime it can recover from perturbations outside the training distribution. Additionally, we introduce a novel transformer-based perception encoder that employs multi-view cross-attention and a learned scene query. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing in the CARLA simulator, as well as showing the ability to handle perturbations in both CARLA and NVIDIA's DRIVE Sim.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
