arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.12771cs.CVcs.GR
收藏

Spectral-GS:用谱熵驯服3D高斯泼溅

Spectral-GS: Taming 3D Gaussian Splatting with Spectral Entropy

Nanjing University(南京大学)

Letian Huang, Jie Guo, Jialin Dan, Ruoyu Fu, Shujie Wang, Yuanqi Li, Yanwen Guo

AI总结:

Spectral-GS 通过谱分析揭示 3D-GS 缺乏形状感知导致针状伪影,提出 3D 形状感知分裂与 2D 视图一致滤波策略,有效消除伪影并增强高频细节表示,实现高质量真实感渲染。

摘要:

最近,3D Gaussian Splatting (3D-GS) 在新视角合成中取得了令人瞩目的结果,展现出高保真度与高效率。然而,它极易产生针状伪影,特别是在提高采样率时。Mip-Splatting 尝试通过用于频率约束的3D平滑滤波器和用于近似超采样的2D Mip滤波器来消除这些伪影。不幸的是,它倾向于产生过度模糊的结果,且有时针状高斯仍然存在。我们对优化和致密化过程中的协方差矩阵进行谱分析,揭示了当前 3D-GS 缺乏形状感知,而是依赖谱半径和视图位置梯度来决定分裂。因此,具有小位置梯度和低谱熵的针状高斯无法分裂,并过拟合高频细节。此外,3D-GS 和 Mip-Splatting 中使用的两种滤波器在放大以合成新视角时都会降低谱熵并增加条件数,导致视图不一致和更明显的伪影。我们基于谱分析的 Spectral-GS 引入了3D形状感知分裂和2D视图一致滤波策略,有效解决了这些问题,增强了 3D-GS 表示高频细节的能力且无明显伪影,实现了高质量的真实感渲染。

英文摘要:

Recently, 3D Gaussian Splatting (3D-GS) has achieved impressive results in novel view synthesis, demonstrating high fidelity and efficiency. However, it easily exhibits needle-like artifacts, especially when increasing the sampling rate. Mip-Splatting tries to remove these artifacts with a 3D smoothing filter for frequency constraints and a 2D Mip filter for approximated supersampling. Unfortunately, it tends to produce over-blurred results, and sometimes needle-like Gaussians still persist. Our spectral analysis of the covariance matrix during optimization and densification reveals that current 3D-GS lacks shape awareness, relying instead on spectral radius and view positional gradients to determine splitting. As a result, needle-like Gaussians with small positional gradients and low spectral entropy fail to split and overfit high-frequency details. Furthermore, both the filters used in 3D-GS and Mip-Splatting reduce the spectral entropy and increase the condition number during zooming in to synthesize novel view, causing view inconsistencies and more pronounced artifacts. Our Spectral-GS, based on spectral analysis, introduces 3D shape-aware splitting and 2D view-consistent filtering strategies, effectively addressing these issues, enhancing 3D-GS's capability to represent high-frequency details without noticeable artifacts, and achieving high-quality photorealistic rendering.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑