FreeSplat：面向室内场景自由视角合成的可泛化3D高斯溅射
FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View Synthesis of Indoor Scenes
AI总结：
针对现有可泛化3D高斯溅射方法视角范围窄、定位不准的问题，提出FreeSplat框架，通过低成本跨视图聚合、像素级三元组融合及自由视角训练策略，实现高效鲁棒的室内宽范围自由视角合成，性能达SOTA。
摘要：
赋予3D高斯溅射（3D Gaussian Splatting）泛化能力颇具吸引力。然而，现有可泛化3D高斯溅射方法因主干网络庞大，大多局限于立体图像间的窄范围插值，因此缺乏准确定位3D高斯、支持宽视角范围自由视角合成的能力。本文提出一种新型框架FreeSplat，能够从长序列输入重建几何一致的3D场景，实现自由视角合成。具体而言，我们首先引入低成本跨视图聚合（Low-cost Cross-View Aggregation），通过在相邻视图间构建自适应代价体，并采用多尺度结构聚合特征来实现。随后，我们提出像素级三元组融合（Pixel-wise Triplet Fusion），以消除重叠视图区域中3D高斯的冗余，并聚合多视图观测到的特征。此外，我们提出一种简单但有效的自由视角训练策略，无论视图数量多少，都能确保在更宽视角范围内实现鲁棒的视图合成。我们的实验结果表明，在不同输入视图数量下，该方法在新视角渲染彩色图质量和深度图精度方面均达到了最先进的新视角合成性能。我们还表明，FreeSplat的推理效率更高，并且能有效减少冗余高斯，为无需深度先验的前馈式大场景重建提供了可能。
英文摘要：
Empowering 3D Gaussian Splatting with generalization ability is appealing. However, existing generalizable 3D Gaussian Splatting methods are largely confined to narrow-range interpolation between stereo images due to their heavy backbones, thus lacking the ability to accurately localize 3D Gaussian and support free-view synthesis across wide view range. In this paper, we present a novel framework FreeSplat that is capable of reconstructing geometrically consistent 3D scenes from long sequence input towards free-view synthesis.Specifically, we firstly introduce Low-cost Cross-View Aggregation achieved by constructing adaptive cost volumes among nearby views and aggregating features using a multi-scale structure. Subsequently, we present the Pixel-wise Triplet Fusion to eliminate redundancy of 3D Gaussians in overlapping view regions and to aggregate features observed across multiple views. Additionally, we propose a simple but effective free-view training strategy that ensures robust view synthesis across broader view range regardless of the number of views. Our empirical results demonstrate state-of-the-art novel view synthesis peformances in both novel view rendered color maps quality and depth maps accuracy across different numbers of input views. We also show that FreeSplat performs inference more efficiently and can effectively reduce redundant Gaussians, offering the possibility of feed-forward large scene reconstruction without depth priors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
