LMAD：面向可解释自动驾驶的集成端到端视觉-语言模型
LMAD: Integrated End-to-End Vision-Language Model for Explainable Autonomous Driving
AI总结：
本文提出LMAD框架，通过引入场景交互与专家适配器，将视觉-语言模型与端到端自动驾驶任务深度对齐，在DriveLM和nuScenes-QA上显著提升驾驶推理性能，推动可解释自动驾驶发展。
摘要：
大型视觉-语言模型（VLMs）在场景理解方面展现出令人期待的能力，提升了驾驶行为的可解释性以及与用户的交互性。现有方法主要基于车载多视角图像和场景推理文本对VLM进行微调，但这种方式往往缺乏自动驾驶所需的整体且细致的场景识别能力以及强大的空间感知能力，尤其是在复杂情境下。为弥补这一不足，我们提出了一种专为自动驾驶设计的新型视觉-语言框架，称为LMAD。我们的框架通过融合全面的场景理解以及带有VLM的、面向特定任务的结构，模拟了现代端到端驾驶范式。特别地，我们在同一驾驶任务结构中引入了初步场景交互和专门的专家适配器，从而更好地使VLM与自动驾驶场景对齐。此外，我们的方法设计为与现有VLM完全兼容，同时能够无缝集成到以规划为导向的驾驶系统中。在DriveLM和nuScenes-QA数据集上的大量实验表明，LMAD显著提升了现有VLM在驾驶推理任务上的性能，为可解释自动驾驶设立了新的标准。
英文摘要：
Large vision-language models (VLMs) have shown promising capabilities in scene understanding, enhancing the explainability of driving behaviors and interactivity with users. Existing methods primarily fine-tune VLMs on on-board multi-view images and scene reasoning text, but this approach often lacks the holistic and nuanced scene recognition and powerful spatial awareness required for autonomous driving, especially in complex situations. To address this gap, we propose a novel vision-language framework tailored for autonomous driving, called LMAD. Our framework emulates modern end-to-end driving paradigms by incorporating comprehensive scene understanding and a task-specialized structure with VLMs. In particular, we introduce preliminary scene interaction and specialized expert adapters within the same driving task structure, which better align VLMs with autonomous driving scenarios. Furthermore, our approach is designed to be fully compatible with existing VLMs while seamlessly integrating with planning-oriented driving systems. Extensive experiments on the DriveLM and nuScenes-QA datasets demonstrate that LMAD significantly boosts the performance of existing VLMs on driving reasoning tasks,setting a new standard in explainable autonomous driving.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
