ZimaBlue：通过可扩展的视频预训练进化出可泛化的世界动作模型
ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training
浏览论文内容
中文总结 AI 辅助
ZimaBlue是一种可扩展框架，通过三阶段训练课程和异步快慢双系统架构，从大规模视频学习可泛化的世界动作模型，在真实机器人零样本评估中使成功率从36.1%提升至77.8%，在多基准测试中表现强劲。
中文摘要 AI 辅助
机器人操纵面临一个根本性的扩展挑战：鲁棒的泛化需要广泛的物理经验，但带动作标签的机器人轨迹收集成本高昂且多样性固有有限。以自我为中心的视频提供了一种更具可扩展性的具身经验来源，其捕获了不同环境中的物体交互、接触动力学、工具使用和长时程行为。核心挑战在于如何将这种丰富但无动作的经验转化为有效的机器人控制。我们提出ZimaBlue，一种用于从大规模视频中学习可泛化世界动作模型（World Action Models，WAMs）的可扩展框架。ZimaBlue遵循三阶段训练课程：首先在大规模人类和机器人以自我为中心的视频上进行因果具身视频预训练，然后通过带有统一动作表示的视频-动作中间训练，将学习到的视觉动力学与异构机器人轨迹建立联系，最后将模型专门化以部署到目标机器人。为了使生成式WAMs适用于实时控制，ZimaBlue还采用了异步快慢双系统架构：高容量的Slow世界模型提供可泛化的时空表示，轻量的Fast分支可在NVIDIA RTX 4090上实现30 Hz的动作预测。在真实机器人的零样本评估中，仅从目标机器人数据扩展到超过120000小时的具身视频，成功率就从36.1%提升至77.8%。ZimaBlue在多个基准测试中还展现出强劲性能，在未见任务上的提升尤为显著。
英文摘要
Robotic manipulation faces a fundamental scaling challenge: robust generalization demands broad physical experience, yet action-labeled robot trajectories are expensive to collect and inherently limited in diversity. Egocentric videos offer a far more scalable source of embodied experience, capturing object interactions, contact dynamics, tool use, and long-horizon behaviors across diverse environments. The central challenge is how to convert this abundant but action-free experience into effective robot control. We introduce ZimaBlue, a scalable framework for learning generalizable World Action Models (WAMs) from large-scale video. ZimaBlue follows a three-stage training curriculum: it first performs causal embodied video pre-training on large-scale human and robot egocentric videos, then grounds the learned visual dynamics in heterogeneous robot trajectories through video-action mid-training with a unified action representation, and finally specializes the model to a target robot for deployment. To make generative WAMs practical for real-time control, ZimaBluefurther adopts an asynchronous Slow-Fast dual-system architecture, where a high-capacity Slow world model provides generalizable spatiotemporal representations and a lightweight Fast branch enables 30 Hz action prediction on NVIDIA RTX 4090. On real-robot zero-shot evaluations, scaling from target-robot data alone to over 120,000 hours of embodied video improves success from 36.1% to 77.8%. ZimaBlue further delivers strong performance across multiple benchmarks, with particularly pronounced gains on unseen tasks.
发表机构
- Joy Future Academy（京东探索研究院）
机构由 AI 辅助整理，请以论文原文为准。
