arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.17540cs.AIcs.RO

TacitVLA:无需推理延迟的视觉-语言-动作模型推理

TacitVLA: Reasoning Vision-Language-Action Models Without Reasoning Latency

NVIDIA(英伟达);New York University(纽约大学);University of California, Berkeley(加州大学伯克利分校);Johns Hopkins University(约翰斯·霍普金斯大学);Stanford University(斯坦福大学)

Jack Lu, Yurong You, Boyi Li, Yulong Cao, Thomas Tian, Chaowei Xiao, Xiangyu Chen, Wenjie Luo, Boris Ivanovic, Marco Pavone, Yan Wang

AI总结:

本文提出TacitVLA,通过让动作专家不依赖当前生成的推理,在保持与显式推理VLA相当性能的同时,最高降低3.4倍延迟,在四个基准上取得优于此前的结果。

摘要:

行动前的文本推理提升了视觉-语言-动作(VLA)模型的性能,但代价是延迟:每个基于推理文本的动作都必须等待自回归解码。本文指出,动作无需等待或依赖生成的推理即可从中受益,因为推理监督已改进了骨干网络表示。我们提出TacitVLA,这是一种推理型VLA,其动作专家从不关注当前调用生成的推理。在长程任务中,推理文本还可作为情景记忆,TacitVLA可在动作执行期间而非行动前生成每个条目。我们在涵盖操纵、自动驾驶、记忆和非记忆任务的四个基准(RoboMME、RMBench、LIBERO、PhysicalAI-AV)上进行受控研究评估。在所有四个基准上,TacitVLA的性能与显式推理对应模型相当,但延迟最高降低3.4倍;与未使用推理训练的相同架构相比,延迟相当,成功率最高提升17.1%。绝对指标方面,它在RoboMME计数套件(82.7% vs 66.8%)和RMBench(77.5% vs 56.5%)上超越了此前最佳已发表结果,在LIBERO-Plus上达到72.9%,在PhysicalAI-AV闭环驾驶挑战公开分割上达到52.5%的场景得分。项目页面:this https URL。

英文摘要:

Textual reasoning before acting has improved vision-language-action (VLA) models, but at the cost of latency: every action conditioned on the reasoning text must wait for autoregressive decoding. We show that actions do not need to wait for or condition on the generated reasoning to benefit from it, because the reasoning supervision already improves the backbone representation. We introduce TacitVLA, a reasoning VLA whose action expert never attends to the reasoning it generates for the current call. On long-horizon tasks where the reasoning text also serves as episodic memory, it can generate each entry during action execution rather than before acting. We evaluate it in a controlled study on four benchmarks spanning manipulation and autonomous driving, memory and non-memory tasks: RoboMME, RMBench, LIBERO, and PhysicalAI-AV. On all four, TacitVLA matches the performance of its explicit-reasoning counterpart at up to 3.4x lower latency, and matches the latency of the same architecture trained without reasoning at up to +17.1% success. In absolute terms, it surpasses the best previously published results on the RoboMME Counting suite (82.7% vs. 66.8%) and RMBench (77.5% vs. 56.5%), and reaches 72.9% on LIBERO-Plus and a 52.5% scene score on the public split of the PhysicalAI-AV closed-loop driving challenge. Project page: this https URL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

相关深度报道

↑