arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.13530cs.CVcs.GR
收藏

L3DG:潜在三维高斯扩散

L3DG: Latent 3D Gaussian Diffusion

Technical University of Munich(慕尼黑工业大学);Meta Reality Labs Zurich(元现实实验室苏黎世分部)

Barbara Roessle, Norman Müller, Lorenzo Porzi, Samuel Rota Bulò, Peter Kontschieder, Angela Dai, Matthias Nießner

AI总结:

本文提出L3DG,首次通过潜在三维高斯扩散实现三维高斯生成建模,利用VQ-VAE压缩潜在空间和稀疏卷积降低扩散复杂度,在提升物体级生成质量的同时支持房间规模场景生成与实时渲染。

摘要:

我们提出L3DG,这是第一种通过潜在三维高斯扩散公式对三维高斯进行生成式三维建模的方法。这使得有效的生成式三维建模成为可能,并可扩展到生成整个房间规模的场景,且这些场景能够非常高效地渲染。为了实现三维高斯的有效合成,我们提出一种潜在扩散公式,在三维高斯的压缩潜在空间中运行。该压缩潜在空间由一个矢量量化变分自编码器(VQ-VAE)学习得到,为此我们采用稀疏卷积架构以高效处理房间规模场景。这样,通过扩散进行的昂贵生成过程的复杂度被大幅降低,从而在物体级生成上获得更高细节,并具备向大型场景扩展的能力。借助三维高斯表示,生成的场景可以从任意视点实时渲染。我们证明,与先前在无条件物体级辐射场合成方面的工作相比,我们的方法显著提升了视觉质量,并展示了其在房间规模场景生成中的适用性。

英文摘要:

We propose L3DG, the first approach for generative 3D modeling of 3D Gaussians through a latent 3D Gaussian diffusion formulation. This enables effective generative 3D modeling, scaling to generation of entire room-scale scenes which can be very efficiently rendered. To enable effective synthesis of 3D Gaussians, we propose a latent diffusion formulation, operating in a compressed latent space of 3D Gaussians. This compressed latent space is learned by a vector-quantized variational autoencoder (VQ-VAE), for which we employ a sparse convolutional architecture to efficiently operate on room-scale scenes. This way, the complexity of the costly generation process via diffusion is substantially reduced, allowing higher detail on object-level generation, as well as scalability to large scenes. By leveraging the 3D Gaussian representation, the generated scenes can be rendered from arbitrary viewpoints in real-time. We demonstrate that our approach significantly improves visual quality over prior work on unconditional object-level radiance field synthesis and showcase its applicability to room-scale scene generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑