arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2511.00917cs.ROcs.AI

Maestro:利用视觉-语言模型编排机器人模块以实现零样本通用机器人

Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robots

Junyao Shi, Rujia Yang, Kaitian Chao, Selina Bingqing Wan, Yifei Shao, Jiahui Lei, Jianing Qian, Long Le, Pratik Chaudhari, Kostas Daniilidis, Chuan Wen, Dinesh Jayaraman

AI总结:

本研究提出Maestro系统,以VLM编码智能体动态组合感知、规划、控制模块生成可编程策略,在零样本操作任务上超越现有VLA模型,且具备高扩展性与适配性。

摘要:

当前通用机器人领域探索最深入的路径,核心是收集规模越来越大的“观测输入-动作输出”机器人数据集来训练大型端到端模型,复刻了已在视觉-语言模型(VLM)中验证有效的方案。我们选择了一条少有人走的路线:直接围绕VLM构建通用策略,通过一组精心筛选的感知、规划与控制模块所封装的特定机器人能力,来增强VLM的通用能力。在Maestro系统中,一个VLM编码智能体会根据当前任务与场景,动态地将这些模块组合成可编程策略。Maestro的架构得益于精简的闭环接口(无过多人工施加的结构约束),以及全面多样的工具库。因此,在具有挑战性的操作技能零样本性能上,它大幅超越了当前的视觉-语言-动作(VLA)模型。此外,Maestro易于扩展以集成新模块,易于编辑以适配新的机器人载体(如搭载机械臂的四足机器人),甚至能通过少量本地代码编辑,从极少的真实世界经验中快速适配。

英文摘要:

Today's best-explored routes towards generalist robots center on collecting ever larger "observations-in actions-out" robotics datasets to train large end-to-end models, copying a recipe that has worked for vision-language models (VLMs). We pursue a road less traveled: building generalist policies directly around VLMs by augmenting their general capabilities with specific robot capabilities encapsulated in a carefully curated set of perception, planning, and control modules. In Maestro, a VLM coding agent dynamically composes these modules into a programmatic policy for the current task and scenario. Maestro's architecture benefits from a streamlined closed-loop interface without many manually imposed structural constraints, and a comprehensive and diverse tool repertoire. As a result, it largely surpasses today's VLA models for zero-shot performance on challenging manipulation skills. Further, Maestro is easily extensible to incorporate new modules, easily editable to suit new embodiments such as a quadruped-mounted arm, and even easily adapts from minimal real-world experiences through local code edits.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑