发表机构
Amazon FAR (Frontier AI & Robotics); The University of Texas at Austin; Carnegie Mellon University（亚马逊前沿人工智能与机器人部门; 德克萨斯大学奥斯汀分校; 卡内基梅隆大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
SimEX通过紧密集成仿真实验，使编码智能体无需演示即可高效获取真实机器人操作技能，仅需10分钟真实交互，显著降低实验成本并提升安全性。
AI 中文摘要
由大型语言模型（LLMs）驱动的编码智能体已在数字世界中展现出自主推理和实现目标的卓越能力。然而，将这一成功带到物理世界仍然具有挑战性。一方面，直接生成方法（如Code as Policies）常常因LLMs对机器人和物理环境理解不足而受到影响。另一方面，在物理世界中进行迭代试错调整（如物理自动研究）会带来显著的实验成本和安全问题。我们引入了SimEX：仿真集成的机器人自动研究，这是一个将仿真实验紧密集成的自动研究框架，使编码智能体能够高效获取控制真实机器人的物理能力。SimEX分两个阶段运行。首先，智能体在仿真中进行开放式探索和优化迭代，开发具有鲁棒性和泛化能力的机器人工具箱。其次，智能体仅通过少量物理试验来同时调整工具箱和仿真器：每次试验修正仿真器，修正后的仿真器用于诊断故障并筛选候选修复方案。我们在仿真到仿真设置以及物理机器人上广泛评估了SimEX。在包括叠毛巾、扫描条形码和操作盘子等具有挑战性的真实世界操作任务中，SimEX使编码智能体无需任何演示，仅需10分钟的真实机器人交互即可高效获取机器人技能。这些结果表明，仿真可以成为实现物理智能的关键组成部分，不仅作为必须紧密复现真实世界的训练数据来源，而且作为一个大致正确的实验室，编码智能体在其中开发执行机器人操作所需的知识和程序。更多细节和机器人视频请访问此https URL。
英文摘要
Coding agents powered by large language models (LLMs) have shown remarkable abilities to autonomously reason about and achieve goals in the digital world. However, bringing this success to the physical world remains challenging. On the one hand, direct generation methods (e.g., Code as Policies) often suffer from the LLMs' insufficient understanding of robots and physical environments. On the other hand, iterative trial-and-error tuning in the physical world (e.g., physical autoresearch) induces significant experimental cost and safety concerns. We introduce SimEX: Simulation-Integrated Robotics AutoResearch, an autoresearch framework that tightly integrates simulated experimentation, enabling coding agents to efficiently acquire physical capabilities for controlling real robots. SimEX operates in two stages. First, the agent conducts open-ended probe-and-optimize iterations in simulation, developing a robot toolbox with robust and generalizable capabilities. Second, the agent adapts the toolbox and the simulator together through only a few physical trials: each trial corrects the simulator, and the corrected simulator is used to diagnose failures and screen candidate repairs. We evaluate SimEX extensively in sim-to-sim settings and on physical robots. On challenging real-world manipulation tasks including towel folding, barcode scanning, and plate manipulation, SimEX enables coding agents to efficiently acquire robot skills without any demonstration and with only 10 minutes of real-robot interaction. These results suggest that simulation can be a critical component in achieving physical intelligence, not only as a source of training data that must closely replicate the real world, but also as a roughly correct laboratory where a coding agent develops the knowledge and procedures needed to act on the robot. More details and robot videos at https://robo-simex.github.io/
