CityRAG: 通过空间感知的视频生成进入城市
CityRAG: Stepping Into a City via Spatially-Grounded Video Generation
AI总结：
CityRAG通过利用地理注册数据的大型语料库，生成空间一致且可导航的真实环境视频，其核心方法是结合学习的先验知识和时空不一致训练数据，以实现复杂的运动和外观变化。
摘要：
我们解决了生成一个空间一致且可导航的环境的问题，该环境是真实位置的模拟。现有的视频生成模型可以产生一个与文本（T2V）或图像（I2V）提示一致的合理序列。然而，能够重建在任意天气条件和动态物体配置下的真实世界对于下游应用如自动驾驶和机器人模拟至关重要。为此，我们提出了CityRAG，一个视频生成模型，利用大规模地理注册数据作为上下文，将生成过程与物理场景结合，同时保持对复杂运动和外观变化的学习先验。CityRAG依赖于时间不一致的训练数据，教会模型将场景的底层属性与瞬时属性语义解耦。我们的实验表明，CityRAG能够生成连贯的分钟级、物理一致的视频序列，保持数千帧的天气和光照条件，实现回环闭合，并导航复杂的轨迹以重建真实世界地理。
英文摘要：
We address the problem of generating a 3D-consistent, navigable environment that is spatially grounded: a simulation of a real location. Existing video generative models can produce a plausible sequence that is consistent with a text (T2V) or image (I2V) prompt. However, the capability to reconstruct the real world under arbitrary weather conditions and dynamic object configurations is essential for downstream applications including autonomous driving and robotics simulation. To this end, we present CityRAG, a video generative model that leverages large corpora of geo-registered data as context to ground generation to the physical scene, while maintaining learned priors for complex motion and appearance changes. CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes. Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
