arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.20577cs.CV
收藏

高效的端到端自动驾驶:通过掩码视觉-语言-动作扩散

Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusion

Institute for Physical Artificial Intelligence (IPAI), Purdue University(物理人工智能研究所(IPAI)、普渡大学);College of Engineering, Purdue University(工程学院、普渡大学);Department of Computer Science, Purdue University(计算机科学系、普渡大学)

Jiaru Zhang, Manav Gagvani, Can Cui, Juntong Peng, Ruqi Zhang, Ziran Wang

AI总结:

MVLAD-AD通过掩码视觉-语言-动作扩散模型,提升自动驾驶的效率与规划精度,实现高效且可解释的端到端自动驾驶。

摘要:

大型语言模型(LLMs)和视觉-语言模型(VLMs)已成为端到端自动驾驶的有希望候选者。然而,这些模型通常在推理延迟、动作精度和可解释性方面面临挑战。现有的自回归方法在逐词生成上速度较慢,而先前的扩散规划器往往依赖于冗长的通用语言标记,这些标记缺乏显式的几何结构。在本工作中,我们提出了用于自动驾驶的掩码视觉-语言-动作扩散(MVLAD-AD),这是一种新的框架,旨在通过掩码视觉-语言-动作扩散模型弥合高效规划与语义可解释性之间的差距。与强制动作进入语言空间的方法不同,我们引入了一种离散动作标记化策略,从现实世界驾驶分布中构建一个紧凑的代码本,包含运动学可行的路径点。此外,我们提出了几何感知的嵌入学习,以确保潜在空间中的嵌入近似物理几何度量。最后,引入了一种优先考虑动作的解码策略,以优先生成轨迹。在nuScenes和衍生基准上的广泛实验表明,MVLAD-AD在效率上表现优异,并在规划精度上优于最先进的自回归和扩散基线,同时提供高保真度和可解释的推理。

英文摘要:

Large Language Models (LLMs) and Vision-Language Models (VLMs) have emerged as promising candidates for end-to-end autonomous driving. However, these models typically face challenges in inference latency, action precision, and explainability. Existing autoregressive approaches struggle with slow token-by-token generation, while prior diffusion-based planners often rely on verbose, general-purpose language tokens that lack explicit geometric structure. In this work, we propose Masked Vision-Language-Action Diffusion for Autonomous Driving (MVLAD-AD), a novel framework designed to bridge the gap between efficient planning and semantic explainability via a masked vision-language-action diffusion model. Unlike methods that force actions into the language space, we introduce a discrete action tokenization strategy that constructs a compact codebook of kinematically feasible waypoints from real-world driving distributions. Moreover, we propose geometry-aware embedding learning to ensure that embeddings in the latent space approximate physical geometric metrics. Finally, an action-priority decoding strategy is introduced to prioritize trajectory generation. Extensive experiments on nuScenes and derived benchmarks demonstrate that MVLAD-AD achieves superior efficiency and outperforms state-of-the-art autoregressive and diffusion baselines in planning precision, while providing high-fidelity and explainable reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑