发表机构
Massachusetts Institute of Technology; National University of Singapore; University of Pennsylvania; NVIDIA（麻省理工学院; 新加坡国立大学; 宾夕法尼亚大学; 英伟达）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
RAPID从单个视觉演示自动生成、验证并改进机器人程序，采用以对象为中心的关系表示，在仿真和真实机器人上均表现优异，泛化能力强。
AI 中文摘要
编码智能体在解决复杂编程问题方面已展现出巨大成功。为了将其潜力应用于机器人系统，本工作引入了从演示中进行机器人智能体编程（RAPID），该方法在给定单个视觉人类演示的情况下，自动生成、验证并改进机器人程序。代码改进的迭代智能体循环需要几个关键要素：（i）可测试的任务规范，（ii）用于机器人执行的动作原语，以及（iii）用于程序执行和验证的交互环境。RAPID自动从演示中推断出所有这三者。为了使生成的程序在演示场景之外可复用，RAPID使用了一种以对象为中心的 relational 程序表示，该表示关注所演示策略的底层结构，而非具体的运动本身：它将动作原语表达为轨迹优化程序，实现对象级运动效果，同时通过关系约束将它们组合起来，在运行时捕获场景特定的几何信息。我们在仿真中评估了RAPID，涉及LIBERO-Pro基准中的八个具有挑战性的接触丰富非抓取操作任务以及一般抓取操作任务。我们还成功将其部署在真实的Franka机械臂上，并评估了所有八个非抓取任务。在所有实验中，RAPID都表现出强大的性能，并在对象姿态、形状、材料和环境方面具有泛化能力。网站：此https URL。
英文摘要
Coding agents have demonstrated enormous success in solving complex programming problems. To leverage their potential for robot systems, this work introduces Robot Agentic Programming from Demonstrations (RAPID), which automatically generates, verifies, and refines robot programs, given a single visual human demonstration. The iterative agentic loop of code refinement requires several key ingredients: (i) a testable task specification, (ii) action primitives for robot execution, and (iii) an interactive environment for program execution and verification. RAPID infers all three from the demonstration automatically. To make the resulting program reusable beyond the demonstration setting, RAPID uses an object-centric relational program representation that focuses on the underlying structure of the demonstrated strategy rather than the specific motion per se: it expresses the action primitives as trajectory-optimization programs that realize object-level motion effects, while composing them through relational constraints that capture scene-specific geometry at run time. We evaluated RAPID in simulation on eight challenging contact-rich nonprehensile manipulation tasks as well as general prehensile manipulation tasks in the LIBERO-Pro benchmark. We also successfully deployed it on a real Franka arm and evaluated on all eight nonprehensile tasks. In all experiments, RAPID demonstrated strong performance, with generalization over object pose, shape, material, and environment. Website: https://yuyaoliu.me/projects/rapid.
