面向可泛化新视角合成的无对极几何3D高斯泼溅
Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis
AI总结：
针对现有可泛化3D高斯泼溅方法依赖不可靠对极先验的问题，本文提出eFreeSplat模型，结合跨视角预训练ViT与迭代跨视角高斯对齐，在宽基线任务上超越现有最优基线。
摘要：
可泛化3D高斯泼溅（3DGS）能够以前馈推理方式从稀疏视角观测重建新场景，无需传统3DGS所需的针对特定场景的重新训练。然而现有方法严重依赖对极先验，这类先验在复杂真实场景中并不可靠，尤其是在非重叠区域和被遮挡区域。本文提出eFreeSplat，这是一种高效的基于前馈3DGS的可泛化新视角合成模型，运行时不依赖对极线约束。为增强具备3D感知的多视角特征提取，我们采用了在大规模数据集上进行跨视角补全预训练的自监督Vision Transformer（ViT，视觉Transformer）。此外，我们提出迭代式跨视角高斯对齐方法，以确保不同视角间的深度尺度一致。eFreeSplat为可泛化新视角合成提供了一种创新思路。与现有纯无几何方法不同，eFreeSplat更侧重通过跨视角预训练提供3D先验，实现无对极约束的特征匹配与编码。我们在RealEstate10K和ACID数据集上的宽基线新视角合成任务中评估了eFreeSplat。大量实验表明，eFreeSplat的表现优于依赖对极先验的现有最优基线方法，在几何重建效果和新视角合成质量上均更出色。项目页面：https://tatakai1.github.io/efreesplat/。
英文摘要：
Generalizable 3D Gaussian splitting (3DGS) can reconstruct new scenes from sparse-view observations in a feed-forward inference manner, eliminating the need for scene-specific retraining required in conventional 3DGS. However, existing methods rely heavily on epipolar priors, which can be unreliable in complex realworld scenes, particularly in non-overlapping and occluded regions. In this paper, we propose eFreeSplat, an efficient feed-forward 3DGS-based model for generalizable novel view synthesis that operates independently of epipolar line constraints. To enhance multiview feature extraction with 3D perception, we employ a selfsupervised Vision Transformer (ViT) with cross-view completion pre-training on large-scale datasets. Additionally, we introduce an Iterative Cross-view Gaussians Alignment method to ensure consistent depth scales across different views. Our eFreeSplat represents an innovative approach for generalizable novel view synthesis. Different from the existing pure geometry-free methods, eFreeSplat focuses more on achieving epipolar-free feature matching and encoding by providing 3D priors through cross-view pretraining. We evaluate eFreeSplat on wide-baseline novel view synthesis tasks using the RealEstate10K and ACID datasets. Extensive experiments demonstrate that eFreeSplat surpasses state-of-the-art baselines that rely on epipolar priors, achieving superior geometry reconstruction and novel view synthesis quality. Project page: https://tatakai1.github.io/efreesplat/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
