arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.15870cs.CV
收藏

Spec-Gaussian:用于3D Gaussian Splatting的各向异性视角依赖外观

Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting

Zhejiang University(浙江大学);The University of Hong Kong(香港大学);ByteDance Inc.(字节跳动公司)

Ziyi Yang, Xinyu Gao, Yangtian Sun, Yihua Huang, Xiaoyang Lyu, Wen Zhou, Shaohui Jiao, Xiaojuan Qi, Xiaogang Jin

AI总结:

本文提出Spec-Gaussian,用ASG外观场替代3D-GS中的SH并结合由粗到细训练,增强对镜面反射与各向异性外观的建模能力。

摘要:

3D Gaussian splatting(3D-GS)的最新进展不仅借助现代GPU光栅化管线实现了实时渲染,还取得了最先进的渲染质量。然而,尽管3D-GS在标准数据集上具有出色的渲染质量和性能,它在准确建模镜面反射和各向异性成分时仍经常遇到困难。该问题源于球谐函数(SH)表示高频信息的能力有限。为克服这一挑战,我们提出Spec-Gaussian,该方法使用各向异性球面Gaussian(ASG)外观场替代SH,以建模每个3D Gaussian的视角依赖外观。此外,我们开发了一种由粗到细的训练策略,以提高学习效率并消除真实场景中由过拟合导致的漂浮物。实验结果表明,我们的方法在渲染质量方面优于现有方法。得益于ASG,我们在不增加3D Gaussian数量的情况下,显著提升了3D-GS对包含镜面反射和各向异性成分的场景进行建模的能力。这一改进扩展了3D-GS处理具有镜面反射和各向异性表面的复杂场景的适用性。项目页面:https://ingra14m.github.io/Spec-Gaussian-website/。

英文摘要:

The recent advancements in 3D Gaussian splatting (3D-GS) have not only facilitated real-time rendering through modern GPU rasterization pipelines but have also attained state-of-the-art rendering quality. Nevertheless, despite its exceptional rendering quality and performance on standard datasets, 3D-GS frequently encounters difficulties in accurately modeling specular and anisotropic components. This issue stems from the limited ability of spherical harmonics (SH) to represent high-frequency information. To overcome this challenge, we introduce Spec-Gaussian, an approach that utilizes an anisotropic spherical Gaussian (ASG) appearance field instead of SH for modeling the view-dependent appearance of each 3D Gaussian. Additionally, we have developed a coarse-to-fine training strategy to improve learning efficiency and eliminate floaters caused by overfitting in real-world scenes. Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality. Thanks to ASG, we have significantly improved the ability of 3D-GS to model scenes with specular and anisotropic components without increasing the number of 3D Gaussians. This improvement extends the applicability of 3D GS to handle intricate scenarios with specular and anisotropic surfaces. Project page is https://ingra14m.github.io/Spec-Gaussian-website/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑