3D Gaussian splatting(3D-GS)的最新进展不仅借助现代GPU光栅化管线实现了实时渲染,还取得了最先进的渲染质量。然而,尽管3D-GS在标准数据集上具有出色的渲染质量和性能,它在准确建模镜面反射和各向异性成分时仍经常遇到困难。该问题源于球谐函数(SH)表示高频信息的能力有限。为克服这一挑战,我们提出Spec-Gaussian,该方法使用各向异性球面Gaussian(ASG)外观场替代SH,以建模每个3D Gaussian的视角依赖外观。此外,我们开发了一种由粗到细的训练策略,以提高学习效率并消除真实场景中由过拟合导致的漂浮物。实验结果表明,我们的方法在渲染质量方面优于现有方法。得益于ASG,我们在不增加3D Gaussian数量的情况下,显著提升了3D-GS对包含镜面反射和各向异性成分的场景进行建模的能力。这一改进扩展了3D-GS处理具有镜面反射和各向异性表面的复杂场景的适用性。项目页面:https://ingra14m.github.io/Spec-Gaussian-website/。
英文摘要:
The recent advancements in 3D Gaussian splatting (3D-GS) have not only facilitated real-time rendering through modern GPU rasterization pipelines but have also attained state-of-the-art rendering quality. Nevertheless, despite its exceptional rendering quality and performance on standard datasets, 3D-GS frequently encounters difficulties in accurately modeling specular and anisotropic components. This issue stems from the limited ability of spherical harmonics (SH) to represent high-frequency information. To overcome this challenge, we introduce Spec-Gaussian, an approach that utilizes an anisotropic spherical Gaussian (ASG) appearance field instead of SH for modeling the view-dependent appearance of each 3D Gaussian. Additionally, we have developed a coarse-to-fine training strategy to improve learning efficiency and eliminate floaters caused by overfitting in real-world scenes. Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality. Thanks to ASG, we have significantly improved the ability of 3D-GS to model scenes with specular and anisotropic components without increasing the number of 3D Gaussians. This improvement extends the applicability of 3D GS to handle intricate scenarios with specular and anisotropic surfaces. Project page is https://ingra14m.github.io/Spec-Gaussian-website/.
SEGS-SLAM: Structure-enhanced 3D Gaussian Splatting SLAM with Appearance Embedding
IRAIS, tjKLIR, College of Artificial Intelligence, Nankai University(IRAIS、tjKLIR、人工智能学院、南开大学);IITRS, Shenzhen Research Institute, Nankai University(IITRS、深圳研究所、南开大学)
Appearance Decomposition Gaussian Splatting for Multi-Traversal Reconstruction
Department of Automation, Key Laboratory of System Control and Information Processing of Ministry of Education, State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis, Shanghai Jiao Tong University(上海交通大学自动化系,系统控制与信息处理教育部重点实验室,航空电子集成与航空系统综合国家重点实验室)
DualDiff3D: Dual Structure-Appearance Diffusion Priors for Reliability-Enhanced 3D Gaussian Splatting
School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Pengcheng Laboratory(鹏城实验室);Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology(广东省超高清沉浸式媒体技术重点实验室)
Underwater360: Reconstructing Underwater Scenes from Panoramic Images with Omnidirectional Gaussian Splatting
School of Software, Dalian University of Technology(大连理工大学软件学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Prometheus Vision Technology Co., Ltd.(天演智视科技有限公司);The Hong Kong University of Science and Technology(香港科技大学)
Micro-macro Wavelet-based Gaussian Splatting for 3D Reconstruction from Unconstrained Images
State Key Laboratory of Complex and Critical Software Environment(复杂与关键软件环境国家重点实验室);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting
Beihang University(北京航空航天大学);Institute of Semiconductors, Chinese Academy of Sciences(中国科学院半导体研究所);Griffith University(格里菲斯大学);RIKEN AIP(理化学研究所人工智能项目);The University of Tokyo(东京大学)
Neural Surface Priors for Editable Gaussian Splatting
IDEAS NCBR;Adam Mickiewicz University(亚当·密茨凯维奇大学);Wrocław University of Science and Technology(弗罗茨瓦夫理工大学);Jagiellonian University(雅盖隆大学);Tooploox;New Jersey Institute of Technology(新泽西理工学院)
Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague(捷克布拉格技术大学捷克信息学、机器人学与控制论研究所);ETH Zurich(苏黎世联邦理工学院);Google DeepMind(谷歌DeepMind);Faculty of Electrical Engineering, Czech Technical University in Prague(捷克布拉格技术大学电气工程学院)
Spatiotemporal Degradation-Aware 3D Gaussian Splatting for Realistic Underwater Scene Reconstruction
School of Astronautics Beihang University Beijing China(航天学院 北航 北京 中国);School of Artificial Intelligence Beihang University Beijing China(人工智能学院 北航 北京 中国);Zhongguancun Academy Beijing China(中关村学院 北京 中国);Beihang University School of Astronautics Beijing China(北航 航天学院 北京 中国);Beihang University(北航);Zhongguancun Academy(中关村学院)
Stereo-GS: Multi-View Stereo Vision Model for Generalizable 3D Gaussian Splatting Reconstruction
Department of Computer Science, Hong Kong Baptist University(香港 Baptist 大学计算机科学系);NVIDIA AI Technology Center, NVIDIA(NVIDIA 人工智能技术中心);School of Control Science and Engineering, Shandong University(控制科学与工程学院,山东大学)
Moving Light Adaptive Colonoscopy Reconstruction via Illumination-Attenuation-Aware 3D Gaussian Splatting
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology(武汉光电国家研究中心,华中科技大学);School of Computer Science, Wuhan University(武汉大学计算机学院);Wuhan United Imaging Healthcare Surgical Technology Co., Ltd(武汉联影医疗手术技术有限公司)