HERMES：用于同时进行3D场景理解与生成的统一自动驾驶世界模型
HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation
AI总结：
提出统一自动驾驶世界模型HERMES，通过BEV表示和世界查询整合3D场景理解与生成，在nuScenes数据集上实现SOTA，生成误差降32.4%，理解指标CIDEr提升8.0%。
摘要：
驾驶世界模型（DWMs）通过实现未来场景预测，已成为自动驾驶的核心。然而，现有DWMs局限于场景生成，未能融入场景理解（即对驾驶环境的解释与推理）。本文提出名为HERMES的统一驾驶世界模型，在驾驶场景中通过统一框架无缝整合3D场景理解与未来场景演化（生成）。具体而言，HERMES利用鸟瞰图（BEV）表示整合多视角空间信息，同时保留几何关系与交互。我们还引入世界查询，通过大语言模型中的因果注意力将世界知识融入BEV特征，为理解与生成任务提供上下文增强。我们在nuScenes和OmniDrive-nuScenes数据集上开展全面研究以验证方法有效性。HERMES实现最先进性能，生成误差降低32.4%，CIDEr等理解指标提升8.0%。模型与代码将公开发布于https://github.com/LMD0311/HERMES。
英文摘要：
Driving World Models (DWMs) have become essential for autonomous driving by enabling future scene prediction. However, existing DWMs are limited to scene generation and fail to incorporate scene understanding, which involves interpreting and reasoning about the driving environment. In this paper, we present a unified Driving World Model named HERMES. We seamlessly integrate 3D scene understanding and future scene evolution (generation) through a unified framework in driving scenarios. Specifically, HERMES leverages a Bird's-Eye View (BEV) representation to consolidate multi-view spatial information while preserving geometric relationships and interactions. We also introduce world queries, which incorporate world knowledge into BEV features via causal attention in the Large Language Model, enabling contextual enrichment for understanding and generation tasks. We conduct comprehensive studies on nuScenes and OmniDrive-nuScenes datasets to validate the effectiveness of our method. HERMES achieves state-of-the-art performance, reducing generation error by 32.4% and improving understanding metrics such as CIDEr by 8.0%. The model and code will be publicly released at https://github.com/LMD0311/HERMES.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
