arXiv 2511.187462026-03-27cs.CVcs.AI

Any4D: 从自然语言和图像生成开放提示的4D生成

Any4D: Open-Prompt 4D Generation from Natural Language and Images

Hao Li, Qiao Sun

备注：: The authors identified issues in the 4D generation pipeline and evaluation that affect result validity. To ensure scientific accuracy, we will revise the methodology and experiments thoroughly before resubmitting. This version should not be cited or relied upon

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Primitive Embodied World Models，通过限制视频生成时间范围，实现语言与视觉表示的细粒度对齐，降低学习复杂度，提升数据效率，并减少推理延迟，支持复杂任务的组合泛化。

摘要：

尽管基于视频生成的具身世界模型受到越来越多关注，但其依赖大规模具身交互数据仍是关键瓶颈。具身数据的稀缺性、收集难度和高维性从根本上限制了语言与动作之间的对齐粒度，并加剧了长视界视频生成的挑战——阻碍生成模型在具身领域实现"GPT时刻"。基于观察：具身数据的多样性远超可能的原始动作空间，我们提出Primitive Embodied World Models (PEWM)，通过限制视频生成到固定较短的视界，本方法1) 实现语言概念与机器人动作视觉表示的细粒度对齐，2) 降低学习复杂度，3) 提高具身数据收集的数据效率，4) 减少推理延迟。通过配备模块化的视觉-语言模型（VLM）规划器和Start-Goal热图引导机制（SGG），PEWM进一步实现了灵活的闭环控制，并支持在扩展复杂任务上原始级策略的组合泛化。本框架利用视频模型中的时空视觉先验和VLM的语义意识，弥合细粒度物理交互与高层推理之间的差距，为可扩展、可解释和通用的具身智能铺平道路。

英文摘要：

While video-generation-based embodied world models have gained increasing attention, their reliance on large-scale embodied interaction data remains a key bottleneck. The scarcity, difficulty of collection, and high dimensionality of embodied data fundamentally limit the alignment granularity between language and actions and exacerbate the challenge of long-horizon video generation--hindering generative models from achieving a \textit{"GPT moment"} in the embodied domain. There is a naive observation: \textit{the diversity of embodied data far exceeds the relatively small space of possible primitive motions}. Based on this insight, we propose \textbf{Primitive Embodied World Models} (PEWM), which restricts video generation to fixed shorter horizons, our approach \textit{1) enables} fine-grained alignment between linguistic concepts and visual representations of robotic actions, \textit{2) reduces} learning complexity, \textit{3) improves} data efficiency in embodied data collection, and \textit{4) decreases} inference latency. By equipping with a modular Vision-Language Model (VLM) planner and a Start-Goal heatmap Guidance mechanism (SGG), PEWM further enables flexible closed-loop control and supports compositional generalization of primitive-level policies over extended, complex tasks. Our framework leverages the spatiotemporal vision priors in video models and the semantic awareness of VLMs to bridge the gap between fine-grained physical interaction and high-level reasoning, paving the way toward scalable, interpretable, and general-purpose embodied intelligence.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。