arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2509.23402cs.CV

WorldSplat:面向自动驾驶的高斯中心前馈4D场景生成

WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Driving

Nankai University(南开大学);Nanjing University, Suzhou(南京大学苏州校区)

Ziyue Zhu, Zhanqian Wu, Zhenxin Zhu, Lijun Zhou, Haiyang Sun, Bing Wan, Kun Ma, Guang Chen, Hangjun Ye, Jin Xie, jian Yang

AI总结:

WorldSplat提出前馈4D场景生成框架,结合4D感知扩散模型与视频细化,生成时空一致的多轨迹驾驶视频,解决生成与重建的矛盾。

摘要:

驾驶场景生成与重建领域的最新进展,通过生成可扩展且可控的训练数据,展现了增强自动驾驶系统的巨大潜力。现有的生成方法主要聚焦于合成多样且高保真的驾驶视频;然而,由于三维一致性的局限和稀疏的视角覆盖,这些方法难以支持便捷且高质量的新视角合成(NVS)。相反,近期的3D/4D重建方法虽显著提升了真实驾驶场景的NVS质量,却本质上缺乏生成能力。为克服场景生成与重建之间的这一困境,我们提出了WorldSplat,一种新颖的用于4D驾驶场景生成的前馈框架。我们的方法通过两个关键步骤有效生成一致的多轨迹视频:(i)引入一个集成多模态信息的4D感知潜在扩散模型,以前馈方式生成像素对齐的4D高斯体。(ii)随后,利用增强的视频扩散模型对从这些高斯体渲染出的新视角视频进行细化。在基准数据集上进行的大量实验表明,WorldSplat能有效生成高保真、时空一致的多轨迹新视角驾驶视频。项目主页:https://wm-research.github.io/worldsplat/

英文摘要:

Recent advances in driving-scene generation and reconstruction have demonstrated significant potential for enhancing autonomous driving systems by producing scalable and controllable training data. Existing generation methods primarily focus on synthesizing diverse and high-fidelity driving videos; however, due to limited 3D consistency and sparse viewpoint coverage, they struggle to support convenient and high-quality novel-view synthesis (NVS). Conversely, recent 3D/4D reconstruction approaches have significantly improved NVS for real-world driving scenes, yet inherently lack generative capabilities. To overcome this dilemma between scene generation and reconstruction, we propose WorldSplat, a novel feed-forward framework for 4D driving-scene generation. Our approach effectively generates consistent multi-track videos through two key steps: (i) We introduce a 4D-aware latent diffusion model integrating multi-modal information to produce pixel-aligned 4D Gaussians in a feed-forward manner. (ii) Subsequently, we refine the novel view videos rendered from these Gaussians using a enhanced video diffusion model. Extensive experiments conducted on benchmark datasets demonstrate that WorldSplat effectively generates high-fidelity, temporally and spatially consistent multi-track novel view driving videos. Project: https://wm-research.github.io/worldsplat/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑