arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.02229cs.CV

FewViewGS:基于少视图匹配与多阶段训练的高斯泼溅

FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training

University of Amsterdam(阿姆斯特丹大学);DUniversum(3DUniversum)

Ruihong Yin, Vladimir Yugay, Yue Li, Sezer Karaoglu, Theo Gevers

AI总结:

本文提出FewViewGS,一种基于3D高斯泼溅的少样本新视角合成方法,通过多阶段训练和匹配一致性约束(无需预训练深度或扩散模型)及局部保持正则化,在稀疏输入下实现准确渲染,性能优于或媲美现有方法。

摘要:

随着神经辐射场(NeRF)以及更近期的3D高斯泼溅(3D Gaussian Splatting)的引入,从图像进行新视角合成(novel view synthesis)的领域取得了快速进展。高斯泼溅因其高效性和准确渲染新视角的能力而被广泛采用。尽管当有足够数量的训练图像时高斯泼溅表现良好,但其非结构化的显式表示在稀疏输入图像场景下容易过拟合,导致渲染性能不佳。为解决这一问题,我们提出了一种基于3D高斯的稀疏输入图像新视角合成方法,该方法能够从训练图像未覆盖的视角准确渲染场景。我们提出了一种多阶段训练方案,在不依赖预训练深度估计或扩散模型的情况下,对新视角施加基于匹配的一致性约束。这是通过利用可用训练图像的匹配来监督在训练帧之间采样的新视角的生成,并辅以颜色、几何和语义损失来实现的。此外,我们引入了一种针对3D高斯的局部保持正则化,通过保持场景的局部颜色结构来消除渲染伪影。在合成和真实世界数据集上的评估表明,在少样本新视角合成方面,我们的方法与现有最先进方法相比具有竞争性或更优的性能。

英文摘要:

The field of novel view synthesis from images has seen rapid advancements with the introduction of Neural Radiance Fields (NeRF) and more recently with 3D Gaussian Splatting. Gaussian Splatting became widely adopted due to its efficiency and ability to render novel views accurately. While Gaussian Splatting performs well when a sufficient amount of training images are available, its unstructured explicit representation tends to overfit in scenarios with sparse input images, resulting in poor rendering performance. To address this, we present a 3D Gaussian-based novel view synthesis method using sparse input images that can accurately render the scene from the viewpoints not covered by the training images. We propose a multi-stage training scheme with matching-based consistency constraints imposed on the novel views without relying on pre-trained depth estimation or diffusion models. This is achieved by using the matches of the available training images to supervise the generation of the novel views sampled between the training frames with color, geometry, and semantic losses. In addition, we introduce a locality preserving regularization for 3D Gaussians which removes rendering artifacts by preserving the local color structure of the scene. Evaluation on synthetic and real-world datasets demonstrates competitive or superior performance of our method in few-shot novel view synthesis compared to existing state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑