WorldCrafter：具有隐式3D感知记忆的一致视频世界模型
WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory
浏览论文内容
中文总结 AI 辅助
WorldCrafter通过可查询的隐式3D感知记忆，实现从单图或文本提示的流式场景探索，显著提升长时一致性与相机控制精度。
中文摘要 AI 辅助
视频世界模型能够实现对动态环境的交互式探索，但在长时间跨度和跨视角下难以尊重先前的观察结果。我们提出了WorldCrafter，一种视频世界模型，它为此目的学习了一种可被相机查询的隐式3D感知记忆。关键洞察在于让请求的视角决定多视角证据如何被压缩到视频生成器有限的令牌预算中。与视频生成器联合训练，一个记忆编码器和姿态条件读出模块在去噪之前将历史观察整合到一组固定的目标视角特定令牌中，无需显式的基于深度的对应关系。通过将此记忆与近期时间上下文和少步蒸馏相结合，WorldCrafter能够从单张输入图像或文本提示实现流式场景探索。在静态和动态场景上的实验表明，在分钟级探索期间，长时间一致性和相机控制精度有显著提升，同时保持了视觉质量。
英文摘要
Video world models enable interactive exploration of dynamic environments, yet struggle to respect prior observations over long horizons and across viewpoints. We present WorldCrafter, a video world model that learns a camera-queryable implicit 3D-aware memory for this purpose. The key insight is to let the requested viewpoint shape how multi-view evidence is compressed into the video generator's limited token budget. Trained jointly with the video generator, a memory encoder and pose-conditioned readout module integrate historical observations into a fixed set of target view-specific tokens before denoising, without explicit depth-based correspondences. By combining this memory with recent temporal context and few-step distillation, WorldCrafter enables streaming scene exploration from a single input image or text prompt. Experiments across static and dynamic scenes show substantial gains in long-horizon consistency and camera-control accuracy while preserving visual quality during minute-scale exploration.
发表机构
- Peking University（北京大学）
- ARC Lab, Tencent IEG（腾讯互动娱乐事业群 ARC 实验室）
机构由 AI 辅助整理，请以论文原文为准。
