arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.01017cs.CVcs.AIcs.LGcs.RO
收藏

Copilot4D:通过离散扩散学习用于自动驾驶的无监督世界模型

Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion

Waabi(瓦比科技);University of Toronto(多伦多大学)

Lunjun Zhang, Yuwen Xiong, Ze Yang, Sergio Casas, Rui Hu, Raquel Urtasun

AI总结:

本文提出Copilot4D,通过VQVAE分词传感器观测并结合离散扩散预测未来,在多个自动驾驶数据集上大幅降低点云预测误差,展示了类GPT无监督学习在机器人领域的潜力。

摘要:

学习世界模型可以以无监督的方式教会智能体世界如何运作。尽管它可以被视为序列建模的一种特例,但在自动驾驶等机器人应用上扩展世界模型的进展,相比使用生成式预训练Transformer(GPT)扩展语言模型的进展要慢一些。我们指出两个原因是主要瓶颈:处理复杂且非结构化的观测空间,以及拥有一个可扩展的生成模型。因此,我们提出Copilot4D,一种新颖的世界建模方法,它首先用VQVAE对传感器观测进行分词,然后通过离散扩散预测未来。为了高效地并行解码和去噪token,我们将掩码生成图像Transformer重构为离散扩散,并通过一些简单的改动对其进行增强,从而带来显著改进。当应用于点云观测上的世界模型学习时,在NuScenes、KITTI Odometry和Argoverse2数据集上,Copilot4D将先前最先进的Chamfer距离在1秒预测中降低了超过65%,在3秒预测中降低了超过50%。我们的结果表明,对分词后的智能体经验进行离散扩散可以释放类似GPT的无监督学习在机器人领域的潜力。

英文摘要:

Learning world models can teach an agent how the world works in an unsupervised manner. Even though it can be viewed as a special case of sequence modeling, progress for scaling world models on robotic applications such as autonomous driving has been somewhat less rapid than scaling language models with Generative Pre-trained Transformers (GPT). We identify two reasons as major bottlenecks: dealing with complex and unstructured observation space, and having a scalable generative model. Consequently, we propose Copilot4D, a novel world modeling approach that first tokenizes sensor observations with VQVAE, then predicts the future via discrete diffusion. To efficiently decode and denoise tokens in parallel, we recast Masked Generative Image Transformer as discrete diffusion and enhance it with a few simple changes, resulting in notable improvement. When applied to learning world models on point cloud observations, Copilot4D reduces prior SOTA Chamfer distance by more than 65% for 1s prediction, and more than 50% for 3s prediction, across NuScenes, KITTI Odometry, and Argoverse2 datasets. Our results demonstrate that discrete diffusion on tokenized agent experience can unlock the power of GPT-like unsupervised learning for robotics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑