RoboBridge：一种用于仿真到现实迁移的自进化具身智能体框架

中文摘要 AI 辅助

将具身智能引入现实世界的一个关键挑战是将能力从仿真迁移到现实，并使智能体在部署后能够持续适应。端到端的视觉-语言-动作策略提供了强大的操作能力，但其向物理环境的迁移通常依赖于校准模拟的视觉和动力学条件、收集额外的目标域演示，并通过进一步训练来优化策略。使用工具的具身智能体提供了灵活的任务编排，但现有系统主要强调在给定环境内的任务执行和经验复用，对跨仿真和现实迁移程序性知识并持续适应的支持有限。我们提出了RoboBridge，一个将仿真到现实迁移视为可执行任务技能持续适应的框架。智能体将任务知识表示为连接任务意图、观察、工具操作和结果验证的程序。利用交互反馈生成候选技能修订，这些修订在持久化或拒绝之前进行评估。一个预训练的视觉-语言-动作策略被暴露为可复用的动作工具，并通过推理时指导增强，无需重新训练底层策略即可实现细粒度执行。RoboBridge将可迁移技能锚定在任务语义和跨仿真与现实共享的交互接口上。这种表示保留了可复用的任务结构，同时允许通过真实世界执行反馈选择性地修订依赖环境的操作。我们在LIBERO-PRO及相应的物理任务上评估了该框架，研究了技能演化和迁移后适应。我们的框架提供了一条从一次性策略部署到跨环境持续程序性学习的路径。

英文摘要

A key challenge in bringing embodied intelligence into the real world is transferring capabilities from simulation to reality and enabling agents to continually adapt after deployment. End-to-end vision-language-action policies provide strong manipulation capabilities, but their transfer to physical environments typically relies on calibrating simulated visual and dynamical conditions, collecting additional target-domain demonstrations, and optimizing the policy through further training. Tool-using embodied agents offer flexible task orchestration, yet existing systems primarily emphasize task execution and experience reuse within a given environment, with limited support for transferring procedural knowledge and continuously adapting it across simulation and reality. We propose RoboBridge, a framework that treats sim-to-real transfer as the continued adaptation of executable task skills. The agent represents task knowledge as procedures connecting task intent, observations, tool operations, and outcome verification. Interaction feedback is used to generate candidate skill revisions, which are evaluated before being persisted or rejected. A pretrained vision-language-action policy is exposed as a reusable action tool and enhanced with inference-time guidance, enabling fine-grained execution without retraining the underlying policy. RoboBridge grounds transferable skills in task semantics and interaction interfaces shared across simulation and reality. This representation preserves reusable task structure while allowing environment-dependent operations to be selectively revised through real-world execution feedback. We evaluate the framework on LIBERO-PRO and corresponding physical tasks, studying both skill evolution and post-transfer adaptation. Our framework provides a route from one-shot policy deployment to continual procedural learning across environments.

发表机构

• Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
• Zhejiang University（浙江大学）
• Beihang University（北京航空航天大学）
• Harbin Institute of Technology（哈尔滨工业大学）
• Shenzhen Institutes of Advanced Technology（中国科学院深圳先进技术研究院）
• Shanghai Jiao Tong University（上海交通大学）

机构由 AI 辅助整理，请以论文原文为准。