评判,然后驾驶:一种以评判为中心的视觉语言动作框架用于自动驾驶
Judge, Then Drive: A Critic-Centric Vision Language Action Framework for Autonomous Driving
AI总结:
本文提出CriticVLA框架,通过多模态评估和单步优化提升自动驾驶决策质量,实验显示其在复杂场景中性能显著优于现有方法。
摘要:
最近视觉语言动作(VLA)模型在自动驾驶中的进展显示出将多模态输入直接映射到控制信号的潜力。然而,先前的VLA方法未明确利用VLA的评判能力来细化驾驶决策,尽管这种能力已在其他LLM领域得到验证,因此限制了其在复杂闭环场景中的表现。本文提出了一种理论启发的两阶段框架CriticVLA,将VLA的作用从执行扩展到评判。CriticVLA首先生成粗略轨迹,然后通过多模态评估和由VLA基础评判器引导的单步优化进行细化,从而获得更高质量的驾驶行为。为支持此过程,我们构建了一个包含1290万条注释轨迹的大规模合成数据集,覆盖多样化的驾驶场景,增强了评判器的推理和细化能力。在Bench2Drive基准上的大量闭环实验表明,CriticVLA显著优于现有最先进基线,实现了73.33%的总成功率,并在挑战性场景中实现了约30%的改进。
英文摘要:
Recent advances in vision language action (VLA) models have shown remarkable potential for autonomous driving by directly mapping multimodal inputs to control signals. However, previous VLA-based methods have not explicitly exploited the critic capability of VLAs to refine driving decisions, even though such capability has been well demonstrated in other LLM-based domains, thereby limiting their performance in complex closed-loop scenarios. In this work, we present a theoretically inspired two-stage framework, CriticVLA, which extends the role of VLAs from acting to judging. CriticVLA first generates a rough trajectory and then refines it through multimodal evaluation and single-step optimization guided by a VLA-based critic, yielding higher-quality driving behaviors. To support this process, we construct a large-scale synthetic dataset of 12.9 million annotated trajectories covering diverse driving scenarios, which enhances the critic's reasoning and refinement abilities. Extensive closed-loop experiments on the Bench2Drive benchmark show that CriticVLA significantly surpasses state-of-the-art baselines, achieving a 73.33% total success rate and delivering about 30% improvement in challenging scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。