arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.11219cs.CV
收藏

Senna-2:对齐视觉语言模型与端到端驾驶策略以实现一致的决策与规划

Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning

Huazhong University of Science & Technology(华中科技大学);Horizon Robotics

Yuehao Song, Shaoyu Chen, Hao Gao, Yifan Zhu, Weixiang Yue, Jialv Zou, Bo Jiang, Zihao Lu, Yu Wang, Qian Zhang, Xinggang Wang

AI总结:

Senna-2通过三阶段训练范式对齐视觉语言模型与端到端驾驶策略,提升决策与规划的一致性,增强驾驶安全性和效率。

摘要:

视觉语言模型(VLMs)通过利用高层语义推理来增强端到端(E2E)驾驶策略的规划能力。然而,现有方法往往忽视了VLM的高层决策与E2E的低层规划之间的双系统一致性。结果,生成的轨迹可能与预期的驾驶决策不一致,导致系统上行指导和决策跟随能力减弱。为了解决这个问题,我们提出了Senna-2,一种先进的VLM-E2E驾驶策略,通过显式对齐两个系统以实现一致的决策与规划。我们的方法遵循一个以一致性为导向的三阶段训练范式。第一阶段,我们进行驾驶预训练以实现初步的决策与规划,决策适配器将VLM的决策以隐式嵌入的形式传输给E2E策略。第二阶段,在开放循环设置下对齐VLM和E2E策略。第三阶段,通过底层的分层强化学习在3DGS环境中进行闭环对齐以增强安全性和效率。广泛的实验表明,Senna-2在双系统一致性方面实现了显著的提升(19.3% F1得分提升),并在开放循环(5.7% FDE减少)和闭环设置(30.6% AF-CR减少)中显著增强了驾驶安全性。

英文摘要:

Vision-language models (VLMs) enhance the planning capability of end-to-end (E2E) driving policy by leveraging high-level semantic reasoning. However, existing approaches often overlook the dual-system consistency between VLM's high-level decision and E2E's low-level planning. As a result, the generated trajectories may misalign with the intended driving decisions, leading to weakened top-down guidance and decision-following ability of the system. To address this issue, we propose Senna-2, an advanced VLM-E2E driving policy that explicitly aligns the two systems for consistent decision-making and planning. Our method follows a consistency-oriented three-stage training paradigm. In the first stage, we conduct driving pre-training to achieve preliminary decision-making and planning, with a decision adapter transmitting VLM decisions to E2E policy in the form of implicit embeddings. In the second stage, we align the VLM and the E2E policy in an open-loop setting. In the third stage, we perform closed-loop alignment via bottom-up Hierarchical Reinforcement Learning in 3DGS environments to reinforce the safety and efficiency. Extensive experiments demonstrate that Senna-2 achieves superior dual-system consistency (19.3% F1 score improvement) and significantly enhances driving safety in both open-loop (5.7% FDE reduction) and closed-loop settings (30.6% AF-CR reduction).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑