发表机构
DAMO Academy, Alibaba Group; Hong Kong Embodied AI Lab; CUHK; Hupan Lab; Alibaba Group; Ant Group（达摩院，阿里巴巴集团; 香港具身人工智能实验室; 香港中文大学; 湖畔实验室; 阿里巴巴集团; 蚂蚁集团）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对机器人学习数据收集瓶颈，提出数字遥操作范式，用生成世界模型解耦数据收集与物理约束。以RynnWorld-Teleop系统实现，含多种技术，能实时生成，训练策略可零样本转移，还能增强数据集，是高保真可扩展数据引擎。
AI 中文摘要
扩展机器人学习需要大量多样的轨迹数据，但目前物理遥操作限制了数据收集，每次演示都将操作员时间与特定硬件和工作空间绑定。我们引入数字遥操作，通过用生成世界模型取代真实机器人，将数据收集与物理约束解耦。在该框架中，操作员的手部姿势流驱动以机器人为中心的生成世界模型，从单个参考图像合成高保真自我中心视频。记录的姿势流可通过标准重定向作为与实体无关的动作标签转移到任何目标机器人，产生用于模仿学习的完整状态动作轨迹，独立于物理硬件。我们在RynnWorld-Teleop中实例化此范式，该系统集成了深度感知骨骼条件、视频扩散变压器上的渐进式人机训练和流式自回归蒸馏。此管道将生成过程压缩为单通道推理，在单个H100 GPU上实现40 + FPS的实时交互生成。仅在RynnWorld-Teleop生成的数据上训练的策略可在各种灵巧的双手任务中实现有效的零样本Sim2Real转移。此外，用我们的数字遥操作数据增强真实世界数据集可持续提高成功率，表明RynnWorld-Teleop是下一代机器人智能体的高保真、可扩展数据引擎。
英文摘要
Scaling robot learning requires massive, diverse trajectory data, yet collection is currently bottlenecked by physical teleoperation, where every demonstration binds operator time to specific hardware and workspaces. We introduce digital teleoperation, a paradigm that decouples data collection from physical constraints by replacing the real robot with a generative world model. In this framework, an operator's hand-pose stream drives a robot-centric generative world model to synthesize high-fidelity egocentric videos from a single reference image. The recorded pose stream serves as an embodiment-agnostic action label transferable to any target robot via standard retargeting, yielding complete state-action trajectories for imitation learning independent of physical hardware. We instantiate this paradigm in RynnWorld-Teleop, a system that integrates depth-aware skeletal conditioning, progressive human-to-robot training on a video Diffusion Transformer (DiT), and streaming autoregressive distillation. This pipeline compresses the generative process into a single-pass inference, enabling 40+ FPS, real-time interactive generation on a single H100 GPU. Policies trained exclusively on RynnWorld-Teleop-generated data achieve effective zero-shot Sim2Real transfer across dexterous and diverse bimanual tasks. Moreover, augmenting real-world datasets with our digitally teleoperated data consistently improves success rates, demonstrating that RynnWorld-Teleop serves as a high-fidelity, scalable data engine for the next generation of robotic agents.
CommentsProject Page: https://alibaba-damo-academy.github.io/RynnWorld-Teleop.github.io, Github: https://github.com/alibaba-damo-academy/RynnWorld-Teleop
