arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.03175cs.CV
收藏

面向城市场景的动态三维高斯场

Dynamic 3D Gaussian Fields for Urban Areas

ETH Zürich(苏黎世联邦理工学院);Meta Reality Labs(元宇宙现实实验室);CTU Prague(布拉格捷克技术大学)

Tobias Fischer, Jonas Kulhanek, Samuel Rota Bulò, Lorenzo Porzi, Marc Pollefeys, Peter Kontschieder

AI总结:

本文提出4DGF,一种结合三维高斯几何骨架与神经场外观模型的大规模动态城市场景表示方法,通过全局场景图与局部形变建模动态,在PSNR上超越现有方法3 dB以上,渲染速度提升超200倍。

摘要:

我们提出了一种高效的神经三维场景表示方法,用于大规模动态城市场景中的新视角合成。现有工作由于视觉质量有限且渲染速度无法交互,难以满足混合现实或闭环仿真等应用需求。近年来,基于光栅化的方法在极高速度下实现了高质量的新视角合成,但这些方法仅限于小规模、同质化数据,无法应对天气、季节和光照引起的显著外观与几何变化,也无法扩展到包含数千张图像的更大规模动态场景。我们提出4DGF,一种可扩展至大规模动态城市场景的神经场景表示方法,能够处理异构输入数据,并大幅提升渲染速度。我们以三维高斯作为高效的几何骨架,同时依赖神经场作为紧凑且灵活的外观模型。我们通过全局尺度的场景图集成场景动态,并在局部层面通过形变建模关节运动。这种分解式方法实现了适合真实世界应用的灵活场景组合。在实验中,我们在PSNR上超越现有最优方法超过3 dB,渲染速度提升超过200倍。

英文摘要:

We present an efficient neural 3D scene representation for novel-view synthesis (NVS) in large-scale, dynamic urban areas. Existing works are not well suited for applications like mixed-reality or closed-loop simulation due to their limited visual quality and non-interactive rendering speeds. Recently, rasterization-based approaches have achieved high-quality NVS at impressive speeds. However, these methods are limited to small-scale, homogeneous data, i.e. they cannot handle severe appearance and geometry variations due to weather, season, and lighting and do not scale to larger, dynamic areas with thousands of images. We propose 4DGF, a neural scene representation that scales to large-scale dynamic urban areas, handles heterogeneous input data, and substantially improves rendering speeds. We use 3D Gaussians as an efficient geometry scaffold while relying on neural fields as a compact and flexible appearance model. We integrate scene dynamics via a scene graph at global scale while modeling articulated motions on a local level via deformations. This decomposed approach enables flexible scene composition suitable for real-world applications. In experiments, we surpass the state-of-the-art by over 3 dB in PSNR and more than 200 times in rendering speed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑