arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.08684cs.CV
收藏

BLUE:迈向自动驾驶高效视觉-语言-动作模型中更好的语言使用

BLUE: Toward Better Language Use in Efficient Vision-Language-Action Models for Autonomous Driving

Bosch Research(博世研究院)

George Ling, Lijin Yang, Hao Yang, Zhongzhan Huang

AI总结:

提出BLUE方法,通过轻量门控机制在视觉-语言-动作模型中按帧决定是否激活语言生成,实现性能提升和2.54倍推理加速。

摘要:

我们提出BLUE,一种在自动驾驶(AD)的视觉-语言-动作(VLA)模型中实现更好语言使用的极简方法。通过广泛分析,我们发现语言仅在一小部分路线上重要,但在这些路线上,语言可以大幅提升或降低性能。因此,在每一帧生成语言是低效的,因为大部分计算花费在无法从语言中受益的帧上。我们进一步表明,预训练的VLA隐藏状态可能已经编码了语言是否会对给定帧有益,尽管场景复杂度和运动特征本身难以预测这一点。基于这一发现,BLUE在冻结的VLA隐藏状态上训练一个轻量级门控,以决定每帧是激活语言生成还是直接预测动作,无需修改主干网络或额外的人工标注。仅用0.11M参数的门控,BLUE在两个基准测试上均达到新的最优水平,在Bench2Drive上实现76.2%的成功率,在Longest6 v2上获得36的驾驶分数,同时相比主干网络实现2.54倍的推理加速和8.9%的成功率提升。BLUE为高效的语言增强自动驾驶提供了一条实用路径,表明VLA模型可以以极低的成本保留语言的优势。我们的代码、数据、日志和检查点完全公开在https://github.com/George-Ling3/BLUE。

英文摘要:

We present BLUE, a minimal method for better language use in vision-language-action (VLA) models for autonomous driving (AD). Through extensive analysis, we reveal that language matters on only a small fraction of routes, but on those routes it can greatly improve or degrade performance. Generating language at every frame is therefore inefficient, since most computation is spent on frames that do not benefit from language. We further show that pretrained VLA hidden states potentially already encode whether language will benefit a given frame, even though scene complexity and kinematic features alone struggle to predict this. Based on this finding, BLUE trains a lightweight gate on frozen VLA hidden states to decide per frame whether to activate language generation or predict actions directly, without modifying the backbone or requiring additional human annotation. With just a 0.11M-parameter gate, BLUE sets a new state of the art on both benchmarks, achieving 76.2% success rate on Bench2Drive and 36 driving score on Longest6 v2, while delivering 2.54x inference speedup and 8.9% success rate improvement over the backbone. BLUE provides a practical path toward efficient language-augmented AD, showing that VLA models can retain the benefits of language at a fraction of the cost. Our code, data, logs and checkpoints are fully available on https://github.com/George-Ling3/BLUE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑