发表机构
University of Bonn; Lamarr Institute for Machine Learning and Artificial Intelligence（波恩大学; 拉马尔机器学习与人工智能研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对3D高斯溅射（3DGS）几何重建不佳的问题，提出融入多视图几何先验并利用其附带的置信度图加权，在标准基准及含镜面物体的复杂场景中实现了重建质量的显著提升。
AI 中文摘要
3D高斯溅射（3DGS）已成为一种广泛应用于新视图合成的工具，能以稀疏表示实现实时渲染。然而，该方法依赖运动恢复结构（SfM）初始化和光度优化，可能导致几何重建效果不佳，尤其对于高镜面反射物体。本研究探讨将预测的法向量图和深度图形式的几何先验融入3DGS框架以提升重建质量。我们分析了将这些先验融入基于GS的方法的效果，评估显示，近期视觉几何基础Transformer（VGGT）所做的多视图预测优于单视图方案，主要原因是估计结果存在置信度图，这是多视图模型的附带产物，可通过适当加权各预测显著提升先验的有效性。在标准基准上开展的大量实验表明，重建质量得到持续提升，且在包含镜面反射物体的复杂场景中获得显著增益。
英文摘要
3D Gaussian splatting (3DGS) has emerged as a widely-used tool for novel view synthesis, offering real-time rendering in a sparse representation. However, the method's reliance on structure-from-motion initialization and photometric optimization can lead to suboptimal geometric reconstruction, particularly for objects with high specularity. In this work, we investigate the integration of geometric priors, in the form of predicted normal and depth maps, into the 3DGS framework to improve the reconstruction quality. We analyze the effect of incorporating these priors into GS-based methods and our evaluation reveals that multi-view predictions, as they are done by the recent visual geometry grounded transformer (VGGT), outperform single-view alternatives. A major factor is the existence of a confidence map for the estimations, which comes as a by-product of multi-view models and which can significantly improve the effectiveness of priors by weighting each prediction appropriately. Extensive experiments on standard benchmarks show consistent improvement in reconstruction quality and significant gains in complex scenes including specular objects.
