MuGS:多基线可泛化高斯泼溅重建
MuGS: Multi-Baseline Generalizable Gaussian Splatting Reconstruction
AI总结:
提出多基线高斯泼溅(MuGS),一种泛化前馈新视角合成方法,融合MVS与MDE特征并设计投影-采样深度融合机制,引入参考视图损失,利用3D高斯表示,在多种基线和场景下达到最先进性能。
摘要:
我们提出了多基线高斯泼溅(MuGS),一种用于新视角合成的泛化前馈方法,能有效处理多样化的基线设置,包括稀疏输入视图中的小基线与大基线情况。具体而言,我们整合了多视图立体(MVS)和单目深度估计(MDE)的特征,以增强可泛化重建的特征表示。接着,我们提出了一种用于深度深度融合的投影-采样机制,该机制构建了一个精细的概率体素,以引导特征图的回归。此外,我们引入了一个参考视图损失,以改善几何和优化效率。我们利用3D高斯表示来加速训练和推理时间,同时提升渲染质量。MuGS在多种基线设置和从简单物体(DTU)到复杂室内外场景(RealEstate10K)的多样场景中均达到了最先进的性能。我们还在LLFF和Mip-NeRF 360数据集上展示了有前景的零样本性能。代码可在https://github.com/EuclidLou/MuGS获取。
英文摘要:
We present Multi-Baseline Gaussian Splatting (MuGS), a generalized feed-forward approach for novel view synthesis that effectively handles diverse baseline settings, including sparse input views with both small and large baselines. Specifically, we integrate features from Multi-View Stereo (MVS) and Monocular Depth Estimation (MDE) to enhance feature representations for generalizable reconstruction. Next, We propose a projection-and-sampling mechanism for deep depth fusion, which constructs a fine probability volume to guide the regression of the feature map. Furthermore, We introduce a reference-view loss to improve geometry and optimization efficiency. We leverage 3D Gaussian representations to accelerate training and inference time while enhancing rendering quality. MuGS achieves state-of-the-art performance across multiple baseline settings and diverse scenarios ranging from simple objects (DTU) to complex indoor and outdoor scenes (RealEstate10K). We also demonstrate promising zero-shot performance on the LLFF and Mip-NeRF 360 datasets. Code is available at https://github.com/EuclidLou/MuGS.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。