SteerVLA：在长尾驾驶场景中引导视觉-语言-动作模型
SteerVLA: Steering Vision-Language-Action Models in Long-Tail Driving Scenarios
AI总结：
SteerVLA通过结合视觉-语言模型的推理能力，生成细粒度语言指令以提升驾驶策略的稳健性和长尾场景下的性能。
摘要：
自动驾驶领域的一个基本挑战是如何将高层的语义推理能力用于长尾事件，同时与低层的反应控制能力相结合以实现稳健驾驶。尽管在大规模网络数据上训练的大型视觉-语言模型（VLMs）提供了强大的常识推理能力，但它们缺乏必要的基础经验来确保车辆控制的安全性。我们提出，一个有效的自主代理应利用VLMs的世界知识，引导可调节的驾驶策略在驾驶场景中实现稳健控制。为此，我们提出了SteerVLA，该方法利用VLMs的推理能力生成细粒度的语言指令，以引导视觉-语言-动作（VLA）驾驶策略。我们方法的关键在于高层VLM与低层VLA之间丰富的语言接口，这使高层策略能够更有效地将推理 grounded 在低层策略的控制输出上。为了提供与车辆控制对齐的细粒度语言监督，我们利用VLM来增强现有的驾驶数据，添加详细的语言注释，我们发现这些注释对于有效的推理和可调节性至关重要。我们评估了SteerVLA在一个具有挑战性的闭环基准上，结果显示其在总体驾驶评分上比最先进的方法高出4.77分，在长尾子集上高出8.04分。项目网站可访问：https://steervla.github.io/。
英文摘要：
A fundamental challenge in autonomous driving is the integration of high-level, semantic reasoning for long-tail events with low-level, reactive control for robust driving. While large vision-language models (VLMs) trained on web-scale data offer powerful common-sense reasoning, they lack the grounded experience necessary for safe vehicle control. We posit that an effective autonomous agent should leverage the world knowledge of VLMs to guide a steerable driving policy toward robust control in driving scenarios. To this end, we propose SteerVLA, which leverages the reasoning capabilities of VLMs to produce fine-grained language instructions that steer a vision-language-action (VLA) driving policy. Key to our method is this rich language interface between the high-level VLM and low-level VLA, which allows the high-level policy to more effectively ground its reasoning in the control outputs of the low-level policy. To provide fine-grained language supervision aligned with vehicle control, we leverage a VLM to augment existing driving data with detailed language annotations, which we find to be essential for effective reasoning and steerability. We evaluate SteerVLA on a challenging closed-loop benchmark, where it outperforms state-of-the-art methods by 4.77 points in overall driving score and by 8.04 points on a long-tail subset. The project website is available at: https://steervla.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
