arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.19657cs.CV

DiffGS: 功能性高斯泼溅扩散

DiffGS: Functional Gaussian Splatting Diffusion

School of Software, Tsinghua University(清华大学软件学院)

Junsheng Zhou, Weiqi Zhang, Yu-Shen Liu

AI总结:

针对3D高斯泼溅生成难的问题,提出基于潜在扩散的DiffGS模型,通过解耦函数表示与离散化算法实现多场景高斯生成,为3DGS建模提供新方向。

摘要:

3D Gaussian Splatting(3DGS,三维高斯泼溅)在渲染速度和保真度方面已展现出令人信服的性能,但由于其离散性和非结构化特性,高斯泼溅的生成仍是一项挑战。在本研究中,我们提出了DiffGS,这是一种基于潜在扩散模型的通用高斯生成器。DiffGS是一款功能强大且高效的3D生成模型,能够生成任意数量的高斯基元,以通过光栅化实现高保真渲染。其核心思路是通过三种新颖的函数对高斯泼溅进行解耦表示,分别用于建模高斯概率、颜色和变换。通过对3DGS的这种新颖解耦,我们用连续的高斯泼溅函数来表示离散且非结构化的3DGS,随后我们训练了一个潜在扩散模型,目标是无条件和有条件地生成这些高斯泼溅函数。同时,我们引入了一种离散化算法,通过八叉树引导的采样和优化,从生成的函数中提取任意数量的高斯。我们探索了DiffGS在多种任务中的应用,包括无条件生成、基于文本、图像和部分3DGS的条件生成,以及点到高斯的生成。我们认为DiffGS为灵活建模和生成高斯泼溅提供了新方向。

英文摘要:

3D Gaussian Splatting (3DGS) has shown convincing performance in rendering speed and fidelity, yet the generation of Gaussian Splatting remains a challenge due to its discreteness and unstructured nature. In this work, we propose DiffGS, a general Gaussian generator based on latent diffusion models. DiffGS is a powerful and efficient 3D generative model which is capable of generating Gaussian primitives at arbitrary numbers for high-fidelity rendering with rasterization. The key insight is to represent Gaussian Splatting in a disentangled manner via three novel functions to model Gaussian probabilities, colors and transforms. Through the novel disentanglement of 3DGS, we represent the discrete and unstructured 3DGS with continuous Gaussian Splatting functions, where we then train a latent diffusion model with the target of generating these Gaussian Splatting functions both unconditionally and conditionally. Meanwhile, we introduce a discretization algorithm to extract Gaussians at arbitrary numbers from the generated functions via octree-guided sampling and optimization. We explore DiffGS for various tasks, including unconditional generation, conditional generation from text, image, and partial 3DGS, as well as Point-to-Gaussian generation. We believe that DiffGS provides a new direction for flexibly modeling and generating Gaussian Splatting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑