3DGS-Enhancer：利用视图一致的2D扩散先验增强无界3D高斯泼溅
3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors
AI总结：
本文提出3DGS-Enhancer，利用2D视频扩散先验将3D视图一致性重构为视频时间一致性问题，通过时空解码器融合增强视角并微调3DGS模型，显著提升稀疏输入下无界场景的新视角渲染质量。
摘要：
新视角合成旨在从多个输入图像或视频中生成场景的新视角，近期诸如3D高斯泼溅（3DGS）等进展已在以高效流程生成照片级真实感渲染方面取得了显著成功。然而，在具有挑战性的设置下（例如稀疏输入视角）生成高质量新视角仍然困难，原因在于欠采样区域信息不足，常常导致明显的伪影。本文提出了3DGS-Enhancer，这是一种用于增强3DGS表示质量的新型流程。我们利用2D视频扩散先验来解决具有挑战性的3D视图一致性问题，将其重新表述为在视频生成过程中实现时间一致性。3DGS-Enhancer恢复所渲染新视角的视图一致潜在特征，并通过时空解码器将其与输入视角进行整合。随后，增强后的视角被用于微调初始3DGS模型，从而显著提升其渲染性能。在大规模无界场景数据集上的广泛实验表明，与最先进的方法相比，3DGS-Enhancer能够产生更优的重建性能和高保真渲染结果。项目网页为https://xiliu8006.github.io/3DGS-Enhancer-project。
英文摘要：
Novel-view synthesis aims to generate novel views of a scene from multiple input images or videos, and recent advancements like 3D Gaussian splatting (3DGS) have achieved notable success in producing photorealistic renderings with efficient pipelines. However, generating high-quality novel views under challenging settings, such as sparse input views, remains difficult due to insufficient information in under-sampled areas, often resulting in noticeable artifacts. This paper presents 3DGS-Enhancer, a novel pipeline for enhancing the representation quality of 3DGS representations. We leverage 2D video diffusion priors to address the challenging 3D view consistency problem, reformulating it as achieving temporal consistency within a video generation process. 3DGS-Enhancer restores view-consistent latent features of rendered novel views and integrates them with the input views through a spatial-temporal decoder. The enhanced views are then used to fine-tune the initial 3DGS model, significantly improving its rendering performance. Extensive experiments on large-scale datasets of unbounded scenes demonstrate that 3DGS-Enhancer yields superior reconstruction performance and high-fidelity rendering results compared to state-of-the-art methods. The project webpage is https://xiliu8006.github.io/3DGS-Enhancer-project .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
