arXiv 2503.076082025-03-10cs.CVcs.RO

AlphaDrive：通过强化学习与推理释放视觉语言模型在自动驾驶中的能力

AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning

Huazhong University of Science and Technology（华中科技大学）；Horizon Robotics（地平线机器人）

Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, Xinggang Wang

备注：: Project Page: https://github.com/hustvl/AlphaDrive

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

AlphaDrive 首次将基于 GRPO 的强化学习与规划推理引入自动驾驶视觉语言模型，通过四项规划奖励和两阶段训练策略显著提升规划性能与训练效率，并展现出涌现的多模态规划能力。

摘要：

OpenAI o1 和 DeepSeek R1 在数学和科学等复杂领域达到甚至超越人类专家水平，其中强化学习（RL）和推理发挥了关键作用。在自动驾驶领域，最近的端到端模型大幅提升了规划性能，但由于常识和推理能力有限，仍然难以应对长尾问题。一些研究将视觉语言模型（VLM）集成到自动驾驶中，但它们通常依赖预训练模型，仅对驾驶数据进行简单的监督微调（SFT），没有进一步探索专门针对规划的训练策略或优化方法。本文提出 AlphaDrive，一个面向自动驾驶中视觉语言模型的强化学习和推理框架。AlphaDrive 引入了四种基于 GRPO 的强化学习奖励，专门针对规划任务设计，并采用结合 SFT 与 RL 的两阶段规划推理训练策略。结果表明，与仅使用 SFT 或不使用推理相比，AlphaDrive 显著提升了规划性能和训练效率。此外，我们还兴奋地发现，经过强化学习训练后，AlphaDrive 展现出一些涌现的多模态规划能力，这对提高驾驶安全性和效率至关重要。据我们所知，AlphaDrive 是首个将基于 GRPO 的强化学习与规划推理集成到自动驾驶中的工作。代码将发布以促进未来研究。

英文摘要：

OpenAI o1 and DeepSeek R1 achieve or even surpass human expert-level performance in complex domains like mathematics and science, with reinforcement learning (RL) and reasoning playing a crucial role. In autonomous driving, recent end-to-end models have greatly improved planning performance but still struggle with long-tailed problems due to limited common sense and reasoning abilities. Some studies integrate vision-language models (VLMs) into autonomous driving, but they typically rely on pre-trained models with simple supervised fine-tuning (SFT) on driving data, without further exploration of training strategies or optimizations specifically tailored for planning. In this paper, we propose AlphaDrive, a RL and reasoning framework for VLMs in autonomous driving. AlphaDrive introduces four GRPO-based RL rewards tailored for planning and employs a two-stage planning reasoning training strategy that combines SFT with RL. As a result, AlphaDrive significantly improves both planning performance and training efficiency compared to using only SFT or without reasoning. Moreover, we are also excited to discover that, following RL training, AlphaDrive exhibits some emergent multimodal planning capabilities, which is critical for improving driving safety and efficiency. To the best of our knowledge, AlphaDrive is the first to integrate GRPO-based RL with planning reasoning into autonomous driving. Code will be released to facilitate future research.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。