智能体作为机器人操作策略
Agent as Policy for Robotic Manipulation
查看机构详情
- University of Notre Dame（圣母大学）
- University of California San Diego（加州大学圣迭戈分校）
- San Diego State University（圣迭戈州立大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本文提出AGP方法，使通用智能体无需任务或环境训练即可直接驱动机器人，通过解释视觉、编写程序、发出命令并修正动作，在多种真实操作任务中实现高成功率，验证了智能体作为机器人策略的可行性。
中文摘要 AI 辅助
我们证明，一个通用智能体可以直接驱动物理机器人在整个任务执行过程中工作，而无需任何针对特定任务或特定环境的训练。我们提出了“智能体作为策略”（AGP）方法，该方法将任务规划与执行置于智能体的控制之下。给定一个任务和一个机器人接口，智能体解释视觉证据、编写可执行程序、发出运动命令，并根据物理结果修正其动作。这使智能体的推理和编程能力得以与物理世界进行持续交互。我们在多个真实世界操作任务中研究了AGP，涵盖精密操作、动态运动和可变形物体。这些任务包括从人类视频中组装、从目标图像进行积木搭建、骰子重新定向、定向投掷以及双臂毛巾折叠。AGP在三种积木搭建配置上分别达到了100%、100%和80%的成功率。这些发现为通用智能体充当机器人策略开辟了道路，通过运行时推理、编程和交互将其自主性扩展到物理操作领域。
英文摘要
We demonstrate that a general-purpose agent can directly drive a physical robot throughout task execution without any task-specific or environment-specific training. We introduce Agent as Policy (AGP), which places task planning and execution under the agent's control. Given a task and a robot interface, the agent interprets visual evidence, writes executable programs, issues motion commands, and revises its actions in response to physical outcomes. This brings the agent's reasoning and programming capabilities into continuous interaction with the physical world. We study AGP across multiple real-world manipulation tasks spanning precision manipulation, dynamic motions, and deformable objects. These include assembly from human videos, block construction from goal images, dice flipping, targeted throwing, and bimanual towel folding. AGP achieves success rates of at least 80% in seven of eight task configurations and significantly outperforms previous agentic robot systems. We further study efficiency through task experience accumulation and find that reusing saved procedures and programs shortens execution time across repeated trials. These findings support a path for general-purpose agents to act as robot policies, extending their autonomy to physical manipulation through runtime reasoning, programming, and interaction.
