3D Gaussian Splatting(3DGS,三维高斯泼溅)在渲染速度和保真度方面已展现出令人信服的性能,但由于其离散性和非结构化特性,高斯泼溅的生成仍是一项挑战。在本研究中,我们提出了DiffGS,这是一种基于潜在扩散模型的通用高斯生成器。DiffGS是一款功能强大且高效的3D生成模型,能够生成任意数量的高斯基元,以通过光栅化实现高保真渲染。其核心思路是通过三种新颖的函数对高斯泼溅进行解耦表示,分别用于建模高斯概率、颜色和变换。通过对3DGS的这种新颖解耦,我们用连续的高斯泼溅函数来表示离散且非结构化的3DGS,随后我们训练了一个潜在扩散模型,目标是无条件和有条件地生成这些高斯泼溅函数。同时,我们引入了一种离散化算法,通过八叉树引导的采样和优化,从生成的函数中提取任意数量的高斯。我们探索了DiffGS在多种任务中的应用,包括无条件生成、基于文本、图像和部分3DGS的条件生成,以及点到高斯的生成。我们认为DiffGS为灵活建模和生成高斯泼溅提供了新方向。
英文摘要:
3D Gaussian Splatting (3DGS) has shown convincing performance in rendering speed and fidelity, yet the generation of Gaussian Splatting remains a challenge due to its discreteness and unstructured nature. In this work, we propose DiffGS, a general Gaussian generator based on latent diffusion models. DiffGS is a powerful and efficient 3D generative model which is capable of generating Gaussian primitives at arbitrary numbers for high-fidelity rendering with rasterization. The key insight is to represent Gaussian Splatting in a disentangled manner via three novel functions to model Gaussian probabilities, colors and transforms. Through the novel disentanglement of 3DGS, we represent the discrete and unstructured 3DGS with continuous Gaussian Splatting functions, where we then train a latent diffusion model with the target of generating these Gaussian Splatting functions both unconditionally and conditionally. Meanwhile, we introduce a discretization algorithm to extract Gaussians at arbitrary numbers from the generated functions via octree-guided sampling and optimization. We explore DiffGS for various tasks, including unconditional generation, conditional generation from text, image, and partial 3DGS, as well as Point-to-Gaussian generation. We believe that DiffGS provides a new direction for flexibly modeling and generating Gaussian Splatting.
Fi-Gaussian: Frequency-Aware Implicit Gaussian Splatting for Single Image Dehazing
College of Mechanical and Vehicle Engineering, Chongqing University(重庆大学机械与车辆工程学院);Department of Electrical and Computer Engineering, Carnegie Mellon University(卡内基梅隆大学电气与计算机工程系);National Innovation Center of Intelligent and Connected Vehicles(国家智能网联汽车创新中心);School of Vehicle and Mobility, Tsinghua University(清华大学车辆与运载学院)
University of Liverpool(利物浦大学);Xi’an Jiaotong-Liverpool University(西交利物浦大学);Universidad Antonio de Nebrija(安东尼奥·德·内布里哈大学);ARIES Research Centre(ARIES研究中心)
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学);Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学);Beijing Academy of Artificial Intelligence, BAAI(北京人工智能研究院,BAAI);Fudan University(复旦大学);Peking University(北京大学)
SA-GS: Scale-Adaptive Gaussian Splatting for Training-Free Anti-Aliasing
Tongji University(同济大学);Ocean University of China(中国海洋大学);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);Duke Kunshan University(昆山杜克大学)
A Mixed-Primitive-based Gaussian Splatting Method for Surface Reconstruction
Lancaster University, United Kingdom(兰卡斯特大学,英国);The University of Queensland, Australia(昆士兰大学,澳大利亚);The Hong Kong Polytechnic University, China(香港理工大学,中国);University at Buffalo, United States of America(布法罗大学,美国)
GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation
University of Southern California(南加州大学);Google(谷歌);Pennsylvania State University(宾夕法尼亚州立大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)
SolidGS: Consolidating Gaussian Surfel Splatting for Sparse-View Surface Reconstruction
Texas A&M University(德克萨斯农工大学);OPPO US Research Center(OPPO美国研究中心);Nanyang Technological University(南洋理工大学);Hong Kong University of Science and Technology(香港科技大学);The University of Hong Kong(香港大学)
PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for 6DoF Object Pose Dataset Generation
Friedrich-Alexander-Universität Erlangen-Nürnberg(弗里德里希-亚历山大-埃尔兰根-纽伦堡大学);National Institute of Advanced Industrial Science and Technology(日本先进工业科学技术研究院);Industrial CPS Research Center(工业网络物理系统研究中心)