通过生成、验证与修订实现智能体式机器人创建
Agentic Robot Creation through Generation, Verification, and Revision
AI总结:
本研究提出智能体式框架Roboforge,结合生成、验证与修订,实现将用户高层意图转化为可运行机器人及对应行为,通过实验验证其有效性并完成物理机器人部署。
摘要:
AI智能体能否将用户的高层意图转化为可运行的机器人并教会其行为?我们从两个互补方面研究该问题:实体创建与行为创建。对于实体创建,我们探究当前多模态模型对机械需求的捕捉能力,并引入计算指标用于制造前评估生成的设计。对于行为创建,我们提出Roboforge,这是一种智能体式框架,通过运动生成、策略学习和基于执行的修订,将任务规范转化为可执行策略。给定机器人实体和任务,智能体生成参考运动、配置学习目标并训练策略。关键在于,运动始终处于循环中:rollout反馈可修订参考运动、奖励或两者。我们在多个MLLMs上评估生成的实体,在四种不同机器人上评估行为创建,对比奖励优化和一次性运动基线。我们的计算指标揭示了生成设计中反复出现的机械故障,而行为实验表明,基于执行的优化可提升任务级行为和执行质量。最后,我们制造了一个智能体生成的大猩猩(Gorilla),并将学习到的行为直接部署到物理机器人上。
英文摘要:
Can an AI agent turn high-level user intent into a functioning robot and teach it how to behave? We study this question from two complementary aspects: embodiment and behavior creation. For embodiment creation, we investigate how well current multimodal models capture mechanical requirements, and introduce computational metrics for evaluating generated designs before fabrication. For behavior creation, we present Roboforge, an agentic framework that turns task specifications into executable policies through motion generation, policy learning, and execution-grounded revision. Given a robot embodiment and task, the agent generates a reference motion, configures the learning objective, and trains a policy. Crucially, motion remains in the loop: rollout feedback can revise the reference motion, reward, or both. We evaluate a generated embodiment across multiple MLLMs and behavior creation across four diverse robots against reward-refinement and one-shot motion baselines. Our computational metrics expose recurring mechanical failures in generated designs, while behavior experiments show that execution-grounded refinement improves task-level behavior and execution quality. Finally, we fabricate an agent-generated Gorilla and directly deploy learned behaviors on the physical robot.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。