BEVDriver：在LLM中利用BEV地图实现鲁棒的闭环驾驶
BEVDriver: Leveraging BEV Maps in LLMs for Robust Closed-Loop Driving
AI总结：
BEVDriver是一种基于LLM的端到端闭环驾驶模型，通过BEV编码器融合多视角图像与LiDAR点云，利用Q-Former将BEV特征与语言指令对齐以规划精确轨迹，在CARLA的LangAuto基准上Driving Score较SoTA提升达18.9%。
摘要：
自动驾驶有望为更高效的未来出行奠定基础，这要求研究领域通过安全、可靠和透明的驾驶来建立信任。大型语言模型（LLM）具备推理能力和自然语言理解能力，展现出作为自车运动规划通用决策者的潜力，能够与人类互动并在为人类驾驶员设计的环境中导航。尽管这一研究方向前景广阔，但当前的自动驾驶方法在结合3D空间定位与LLM的推理和语言能力方面仍面临挑战。我们提出了BEVDriver，这是一种基于LLM的端到端闭环驾驶模型，在CARLA中利用潜在的BEV特征作为感知输入。BEVDriver包含一个BEV编码器，可高效处理多视角图像和3D LiDAR点云。在共同的潜在空间中，BEV特征通过Q-Former传播以与自然语言指令对齐，并传递给LLM，后者在考虑导航指令和关键场景的同时预测并规划精确的未来轨迹。在LangAuto基准上，与SoTA方法相比，我们的模型在Driving Score上性能提升高达18.9%。
英文摘要：
Autonomous driving has the potential to set the stage for more efficient future mobility, requiring the research domain to establish trust through safe, reliable and transparent driving. Large Language Models (LLMs) possess reasoning capabilities and natural language understanding, presenting the potential to serve as generalized decision-makers for ego-motion planning that can interact with humans and navigate environments designed for human drivers. While this research avenue is promising, current autonomous driving approaches are challenged by combining 3D spatial grounding and the reasoning and language capabilities of LLMs. We introduce BEVDriver, an LLM-based model for end-to-end closed-loop driving in CARLA that utilizes latent BEV features as perception input. BEVDriver includes a BEV encoder to efficiently process multi-view images and 3D LiDAR point clouds. Within a common latent space, the BEV features are propagated through a Q-Former to align with natural language instructions and passed to the LLM that predicts and plans precise future trajectories while considering navigation instructions and critical scenarios. On the LangAuto benchmark, our model reaches up to 18.9% higher performance on the Driving Score compared to SoTA methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
