驾驶场景的具身理解
Embodied Understanding of Driving Scenarios
AI总结：
针对现有视觉语言模型缺乏空间感知和长时程推演能力的问题，研究提出专为大时空跨度驾驶场景理解设计的ELM框架，经多维度基准验证性能全面超越此前最优方法，相关资源将公开。
摘要：
具身场景理解是自主智能体感知、解读并响应开放驾驶场景的基石。这类理解通常建立在视觉语言模型（Vision-Language Models, VLMs）之上。然而，现有VLM局限于二维领域，缺乏空间感知能力与长时程推演能力。我们重新审视自动驾驶的核心维度，并制定了相应的评估标准。据此，我们提出具身语言模型（Embodied Language Model, ELM）——这是一个专为智能体理解大时空跨度驾驶场景设计的综合框架。ELM引入空间感知预训练，赋予智能体强大的空间定位能力；此外，该模型采用时间感知令牌选择机制，可精准查询时间线索。我们在重构的多维度基准上对ELM进行了实例化验证，其在所有维度上均超越了此前的最优方法。所有代码、数据与模型都将公开共享。
英文摘要：
Embodied scene understanding serves as the cornerstone for autonomous agents to perceive, interpret, and respond to open driving scenarios. Such understanding is typically founded upon Vision-Language Models (VLMs). Nevertheless, existing VLMs are restricted to the 2D domain, devoid of spatial awareness and long-horizon extrapolation proficiencies. We revisit the key aspects of autonomous driving and formulate appropriate rubrics. Hereby, we introduce the Embodied Language Model (ELM), a comprehensive framework tailored for agents' understanding of driving scenes with large spatial and temporal spans. ELM incorporates space-aware pre-training to endow the agent with robust spatial localization capabilities. Besides, the model employs time-aware token selection to accurately inquire about temporal cues. We instantiate ELM on the reformulated multi-faced benchmark, and it surpasses previous state-of-the-art approaches in all aspects. All code, data, and models will be publicly shared.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
