arXiv 2601.064742026-01-17cs.CVcs.AI

SparseOccVLA: 通过稀疏查询弥合占用与视觉语言模型之间的鸿沟，实现统一的4D场景理解和规划

SparseOccVLA: Bridging Occupancy and Vision-Language Models via Sparse Queries for Unified 4D Scene Understanding and Planning

Huazhong University of Science and Technology（华中科技大学）；Xiaomi EV（小米电动车）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）

Chenxu Dang, Jie Wang, Guang Li, Zhiwen Hou, Zihan You, Hangjun Ye, Jie Ma, Long Chen, Yan Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

SparseOccVLA通过稀疏查询整合视觉语言模型与语义占用，实现统一的4D场景理解和规划，提升自动驾驶性能。

摘要：

在自动驾驶领域，视觉语言模型（VLMs）在高层推理方面表现出色，而语义占用提供了细粒度的细节。尽管在各自领域取得了显著进展，但仍没有方法能够有效整合这两种范式。传统VLMs在令牌爆炸和时空推理有限方面存在困难，而语义占用提供了一个统一的显式空间表示，但过于密集，难以与VLMs高效整合。为了解决这些挑战并弥合VLMs与占用之间的差距，我们提出了SparseOccVLA，一种新的视觉语言-动作模型，通过稀疏占用查询统一了场景理解、占用预测和轨迹规划。从轻量级的稀疏占用编码器开始，SparseOccVLA生成紧凑且信息丰富的稀疏占用查询，作为视觉与语言之间的单一桥梁。这些查询被对齐到语言空间，并通过LLM进行推理，实现统一的场景理解和未来占用预测。此外，我们引入了由LLM引导的锚点扩散规划器，具有解耦的锚点评分和去噪功能，以及跨模型轨迹条件融合。SparseOccVLA在OmniDrive-nuScenes上实现了比现有最佳方法在CIDEr上的7%相对提升，在Occ3D-nuScenes上实现了mIoU分数的0.5提升，并在nuScenes基准上设定了开环规划指标的新纪录，展示了其强大的整体能力。

英文摘要：

In autonomous driving, Vision Language Models (VLMs) excel at high-level reasoning , whereas semantic occupancy provides fine-grained details. Despite significant progress in individual fields, there is still no method that can effectively integrate both paradigms. Conventional VLMs struggle with token explosion and limited spatiotemporal reasoning, while semantic occupancy provides a unified, explicit spatial representation but is too dense to integrate efficiently with VLMs. To address these challenges and bridge the gap between VLMs and occupancy, we propose SparseOccVLA, a novel vision-language-action model that unifies scene understanding, occupancy forecasting, and trajectory planning powered by sparse occupancy queries. Starting with a lightweight Sparse Occupancy Encoder, SparseOccVLA generates compact yet highly informative sparse occupancy queries that serve as the single bridge between vision and language. These queries are aligned into the language space and reasoned by the LLM for unified scene understanding and future occupancy forecasting. Furthermore, we introduce an LLM-guided Anchor-Diffusion Planner featuring decoupled anchor scoring and denoising, as well as cross-model trajectory-condition fusion. SparseOccVLA achieves a 7% relative improvement in CIDEr over the state-of-the-art on OmniDrive-nuScenes, a 0.5 increase in mIoU score on Occ3D-nuScenes, and sets state-of-the-art open-loop planning metric on nuScenes benchmark, demonstrating its strong holistic capability.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。