LivingWorld:基于环境动态的交互式4D世界生成
LivingWorld: Interactive 4D World Generation with Environmental Dynamics
AI总结:
本文提出LivingWorld框架,通过单张图像生成具有环境动态的4D世界,解决大规模环境动态建模中的一致性与低延迟问题,采用几何感知对齐模块和紧凑哈希运动场实现高效动态传播。
摘要:
我们介绍了LivingWorld,一种从单张图像生成具有环境动态的交互式4D世界的框架。尽管近期3D场景生成技术能实现大规模环境创建,但大多数方法主要关注静态几何重建,未探索如云、水或烟等场景级环境动态。建模此类动态具有挑战性,因为运动必须在扩展场景中保持一致,同时支持低延迟用户反馈。LivingWorld通过逐步构建全局一致的运动场来解决这一挑战。为维持扩展过程中的全局一致性,我们引入了几何感知对齐模块,以解决跨视角的方向和尺度模糊性。我们进一步使用紧凑的哈希基运动场表示运动,实现高效查询和稳定传播动态。这种表示也支持渲染期间的双向运动传播,产生长且时间一致的4D序列,无需依赖昂贵的视频基细化。在单个RTX 5090 GPU上,生成每个新场景扩展步骤需要9秒,随后3秒用于运动对齐和运动场更新,实现了具有全局一致环境动态的交互式4D世界生成。视频演示可在cvsp-lab.github.io/LivingWorld上查看。
英文摘要:
We introduce LivingWorld, an interactive framework for generating 4D worlds with environmental dynamics from a single image. While recent advances in 3D scene generation enable large-scale environment creation, most approaches focus primarily on reconstructing static geometry, leaving scene-scale environmental dynamics such as clouds, water, or smoke largely unexplored. Modeling such dynamics is challenging because motion must remain coherent across an expanding scene while supporting low-latency user feedback. LivingWorld addresses this challenge by progressively constructing a globally coherent motion field as the scene expands. To maintain global consistency during expansion, we introduce a geometry-aware alignment module that resolves directional and scale ambiguities across views. We further represent motion using a compact hash-based motion field, enabling efficient querying and stable propagation of dynamics throughout the scene. This representation also supports bidirectional motion propagation during rendering, producing long and temporally coherent 4D sequences without relying on expensive video-based refinement. On a single RTX 5090 GPU, generating each new scene expansion step requires 9 seconds, followed by 3 seconds for motion alignment and motion field updates, enabling interactive 4D world generation with globally coherent environmental dynamics. Video demonstrations are available at paper.pnu-cvsp.com/LivingWorld.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。