arXiv 2508.208402025-11-24cs.ROcs.AIcs.MM

学习原始具身世界模型：迈向可扩展的机器人学习

Learning Primitive Embodied World Models: Towards Scalable Robotic Learning

Shanghai AI Lab（上海人工智能实验室）；Fudan（复旦大学）；SJTU（上海交通大学）；NJUST（南京理工大学）；THU（清华大学）；Harvard（哈佛大学）；ZJU（浙江大学）；NJU（南京大学）；USTC（中国科学技术大学）；Tongji（同济大学）；HKUST (GZ)（香港科技大学（广州））

Qiao Sun, Liujia Yang, Wei Tang, Wei Huang, Kaixin Xu, Yongchao Chen, Mingyu Liu, Jiange Yang, Haoyi Zhu, Yating Wang, Tong He, Yilun Chen, Xili Dai, Nanyang Ye, Qinying Gu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出原始具身世界模型（PEWM）以解决具身数据稀疏和高维问题，通过短视界视频生成实现细粒度对齐、降低学习复杂度并提高数据效率。

摘要：

尽管基于视频生成的具身世界模型越来越受到关注，但其对大规模具身交互数据的依赖仍然是一个关键瓶颈。具身数据的稀缺性、收集难度以及高维性从根本上限制了语言与动作之间的对齐粒度，并加剧了长视界视频生成的挑战——阻碍生成模型在具身领域实现“GPT时刻”。有一个直观的观察：具身数据的多样性远超相对较小的可能原始动作空间。基于这一见解，我们提出了一种新的世界建模范式——原始具身世界模型（PEWM）。通过将视频生成限制在固定的短视界范围内，我们的方法1）使语言概念与机器人动作的视觉表示之间实现细粒度对齐，2）降低学习复杂性，3）提高具身数据收集的数据效率，4）减少推理延迟。通过配备模块化的视觉-语言模型（VLM）规划器和起始-目标热图指导机制（SGG），PEWM进一步实现了灵活的闭环控制，并支持在复杂任务中对原始级策略的组合泛化。我们的框架利用视频模型中的时空视觉先验和VLM的语义意识，弥合了细粒度物理交互与高层推理之间的差距，为可扩展、可解释和通用的具身智能铺平了道路。

英文摘要：

While video-generation-based embodied world models have gained increasing attention, their reliance on large-scale embodied interaction data remains a key bottleneck. The scarcity, difficulty of collection, and high dimensionality of embodied data fundamentally limit the alignment granularity between language and actions and exacerbate the challenge of long-horizon video generation--hindering generative models from achieving a "GPT moment" in the embodied domain. There is a naive observation: the diversity of embodied data far exceeds the relatively small space of possible primitive motions. Based on this insight, we propose a novel paradigm for world modeling--Primitive Embodied World Models (PEWM). By restricting video generation to fixed short horizons, our approach 1) enables fine-grained alignment between linguistic concepts and visual representations of robotic actions, 2) reduces learning complexity, 3) improves data efficiency in embodied data collection, and 4) decreases inference latency. By equipping with a modular Vision-Language Model (VLM) planner and a Start-Goal heatmap Guidance mechanism (SGG), PEWM further enables flexible closed-loop control and supports compositional generalization of primitive-level policies over extended, complex tasks. Our framework leverages the spatiotemporal vision priors in video models and the semantic awareness of VLMs to bridge the gap between fine-grained physical interaction and high-level reasoning, paving the way toward scalable, interpretable, and general-purpose embodied intelligence.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。