发表机构
Massachusetts Institute of Technology（麻省理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出RIVET框架，通过共享对象中心表示引导VLM生成可重用操作程序，在真实机器人上实现83%成功率，适应不同几何与关系任务。
AI 中文摘要
构建机器人操作系统需要将感知、规划和控制通过精心设计的表示和接口连接起来。VLM代码生成提供了一种自动化构建该系统的途径，但独立生成的组件可能在几何和任务级信息上不兼容。我们提出了表示引导的VLM生成可执行任务程序集成框架（RIVET），该框架围绕共享的以对象为中心的表示来生成完整的操作系统。该表示结合了每个对象的6D姿态（保留了动作落地所需的度量信息）与关系图（暴露了规划所需的任务级结构）。在该表示的引导下，VLM生成协作的感知、渲染、关系推断和规划程序，每个程序将任务特定计算与可用的包相结合。生成的程序为操作领域编写一次，并在未见过的起始和目标配置上重用，无需重新生成代码。我们在仿真和物理机器人上评估了RIVET在立方体堆叠、七巧板重排和三维装配中的表现，通过重用离线生成的系统，在真实世界中实现了83%的总体成功率。我们的结果表明，表示引导的程序生成可以将通用操作框架适应于具有不同几何、关系和顺序需求的任务。
英文摘要
Building a robotic manipulation system requires connecting perception, planning, and control through carefully designed representations and interfaces. VLM code generation offers a way to automate this construction, but independently generated components may operate on incompatible geometric and task-level information. We present Representation-guided Integration of VLM-generated Executable Task programs (RIVET), a framework for generating complete manipulation systems around a shared object-centric representation. The representation combines per-object 6D poses, which preserve the metric information required for action grounding, with a relation graph that exposes the task-level structure required for planning. Guided by this representation, a VLM generates cooperating perception, rendering, relation-inference, and planning programs, each combining task-specific computation with available packages where useful. The resulting programs are authored once for a manipulation domain and reused on unseen start and goal configurations without code regeneration. We evaluate RIVET on cube stacking, tangram rearrangement, and three-dimensional assembly in simulation and on a physical robot, where we achieve 83% overall success rate in the real world by reusing offline-generated systems. Our results demonstrate that representation-guided program generation can adapt a common manipulation framework to tasks with different geometric, relational, and sequential requirements.
