arXiv 2606.124022026-06-11cs.ROcs.AIcs.CV

DIRECT: 在具身规划器中何时何地分配测试时计算？

DIRECT: When and Where Should You Allocate Test-Time Compute in Embodied Planners?

Stanford University（斯坦福大学）；University of Waterloo（滑铁卢大学）；NVIDIA（英伟达）

Jadelynn Dao, Milan Ganai, Yasmina Abukhadra, Ajay Sridhar, Mozhgan Nasr Azadani, Katie Luo, Clark Barrett, Jiajun Wu, Chelsea Finn, Marco Pavone

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出DIRECT路由框架，根据多模态场景上下文按提示分配计算资源，优化成功-成本帕累托前沿，实验表明不同缩放轴带来不同能力增益，在物理机器人上以更低延迟匹配或超越更强模型。

摘要：

视觉语言模型（VLM）越来越多地被部署为具身智能体的高层规划器，一种新兴策略是扩展测试时计算以提高能力。然而，我们观察到这样做会增加延迟、令牌使用和FLOPs，同时在下游任务中产生不均匀且往往递减的收益，限制了具身智能体的部署范围。我们认为，选择何时何地花费测试时计算是将前沿性能带入现实世界的关键。我们引入了DIRECT，一个路由框架，利用多模态场景上下文按提示分配计算资源，在固定模型选择上改进了成功-成本帕累托前沿。在三种主要的缩放轴（即思维链深度、模型大小和记忆历史）上，我们在VLABench和RoboMME上的实验表明，测试时计算并非均匀的杠杆：不同的轴产生性质不同的能力增益。我们在DROID设置中的物理Franka机械臂上验证了这些见解，涵盖了零样本操作和长程链式任务，我们的路由器以高达65%的平均延迟降低匹配或超过了更强模型的成功率。最终，我们的结果表明，天真地扩展测试时计算是浪费的，而DIRECT能够以极低的成本在机器人系统中提供前沿级别的具身规划。项目页面可在此http URL找到。

英文摘要：

Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents, with an emerging strategy of scaling test-time compute to improve capability. However, we observe that doing so increases latency, token usage, and FLOPs while yielding uneven, often diminishing gains in downstream success, limiting where embodied agents can be deployed. We argue that choosing when and where to spend test-time compute is central to bringing frontier performance to the real world. We introduce DIRECT, a routing framework that uses multimodal scene context to allocate compute per prompt, improving the success--cost Pareto frontier over fixed model selection. Across three dominant scaling axes, namely chain-of-thought depth, model size, and memory history, our experiments on VLABench and RoboMME show that test-time compute is not a uniform lever: different axes yield qualitatively distinct capability gains. We validate these insights on a physical Franka arm in a DROID setup spanning zero-shot manipulation and long-horizon chaining, where our router matches or exceeds a stronger model's success rate at up to 65% lower average latency. Ultimately, our results show that naively scaling test-time compute is wasteful, and that DIRECT can provide frontier-level embodied planning in robotic systems at a fraction of the cost. Project page can be found at jadee-dao.github.io/direct/.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。