arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.32028cs.RO

DVG-WM:解耦视频生成实现高效机器人操作具身世界模型

DVG-WM: Disentangled Video Generation Enables Efficient Embodied World Model for Robotic Manipulation

Nanyang Technological University, Singapore(南洋理工大学(新加坡));Beijing University of Posts and Telecommunications, Beijing, China(北京邮电大学(中国北京));GigaAI(极佳科技)

Ziyu Shan, Zhenyu Wu, Xiaofeng Wang, Zheng Zhu, Ziwei Wang

AI总结:

提出解耦视频生成世界模型(DVG-WM),将世界模型分解为动力学学习和视觉合成,通过流匹配直接映射动力学到视频潜变量,并引入潜变量退化机制再生接触细节,在LIBERO和真实平台实现高达3.97倍加速。

摘要:

基于视频的具身世界模型通过预测未来状态为机器人操作提供了有吸引力的基础,但当前方法仍受限于一个基本纠缠:精确建模动力学通常需要低层时间推理,而生成高分辨率帧则需要根据高层语义进行扩展的视觉合成。这种纠缠导致迭代规划推理速度慢,或预测过于粗糙而无法保留接触丰富的细节。为解决这一困境,我们提出了解耦视频生成世界模型(DVG-WM),一个高效框架,明确将世界模型分解为动力学学习和视觉合成。以初始观察和语言指令为条件,我们的模型首先生成合理的中间视觉状态序列以预览物理交互,并对其进行细化以获得高保真视频。此外,提出了一种高效的级联机制,其中DVG-WM使用流匹配直接将动力学映射到视频潜变量,并引入潜变量退化机制以再生接触丰富的细节。在LIBERO和真实平台上的实验表明,视频质量提升且加速高达3.97倍,验证了解耦视频生成可以作为机器人操作的高效具身世界模型。

英文摘要:

Video-based embodied world models provide an appealing substrate for robotic manipulation by predicting future states, yet current approaches remain limited by a fundamental entanglement: accurately modeling dynamics typically requires low-level temporal reasoning, while producing high-resolution frames demands expansive visual synthesis according to high-level semantics. This entanglement results in slow inference speed for iterative planning or too coarse predictions to retain contact-rich details. To solve this dilemma, we present Disentangled Video Generation World Model (DVG-WM), an efficient framework that explicitly decomposes world modeling into dynamics learning and visual synthesis. Conditioned on an initial observation and a language instruction, our model first generates a plausible sequence of intermediate visual states to preview the physical interaction and refines them to obtain high-fidelity videos. Furthermore, an efficient cascading mechanism is proposed, where DVG-WM uses flow matching to directly map the dynamics to video latents, and introduces a latent degradation mechanism to regenerate contact-rich details. Experiments on LIBERO and real-world platforms demonstrate improved video quality with up to 3.97 times acceleration, validating that disentangled video generation can be an efficient embodied world model for robotic manipulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑