SparSplat：基于可泛化二维高斯泼溅的快速多视图重建
SparSplat: Fast Multi-View Reconstruction with Generalizable 2D Gaussian Splatting
AI总结：
本文提出SparSplat，一种基于MVS学习框架前馈回归2DGS表面元素参数的方法，联合解决稀疏视图下的三维重建与新视角合成，在DTU等基准上取得最先进结果且推理速度提升近两个数量级。
摘要：
通过多视图立体重建（MVS）和新视角合成（NVS）从场景中恢复三维信息本身具有挑战性，尤其是在涉及稀疏视图设置的场景中。三维高斯泼溅（3DGS）的出现实现了实时、照片级真实感的新视角合成。随后，二维高斯泼溅（2DGS）利用透视精确的二维高斯图元光栅化在渲染过程中实现精确的几何表示，在保持实时性能的同时改进了三维场景重建。近期方法已在可泛化的、基于MVS的学习框架内使用3DGS解决稀疏实时NVS问题，以回归三维高斯参数。我们的工作通过联合解决可泛化稀疏三维重建与NVS的挑战，扩展了这一研究方向，并成功地在两项任务上均取得了良好表现。我们提出了一种基于MVS的学习流程，以前馈方式回归2DGS表面元素参数，从而从稀疏视图图像执行三维形状重建和NVS。我们进一步表明，我们的可泛化流程可以受益于已有的基础性多视图深度视觉特征。所得模型在DTU稀疏三维重建基准上以到真值的Chamfer距离衡量取得了最先进结果，同时在新视角合成方面也达到了最先进水平。该模型还在BlendedMVS和Tanks and Temples数据集上展示了强大的泛化能力。我们注意到，我们的模型在基于隐式表示体渲染的前馈稀疏视图重建方面优于先前的最先进方法，同时提供了几乎两个数量级的推理速度提升。
英文摘要：
Recovering 3D information from scenes via multi-view stereo reconstruction (MVS) and novel view synthesis (NVS) is inherently challenging, particularly in scenarios involving sparse-view setups. The advent of 3D Gaussian Splatting (3DGS) enabled real-time, photorealistic NVS. Following this, 2D Gaussian Splatting (2DGS) leveraged perspective accurate 2D Gaussian primitive rasterization to achieve accurate geometry representation during rendering, improving 3D scene reconstruction while maintaining real-time performance. Recent approaches have tackled the problem of sparse real-time NVS using 3DGS within a generalizable, MVS-based learning framework to regress 3D Gaussian parameters. Our work extends this line of research by addressing the challenge of generalizable sparse 3D reconstruction and NVS jointly, and manages to perform successfully at both tasks. We propose an MVS-based learning pipeline that regresses 2DGS surface element parameters in a feed-forward fashion to perform 3D shape reconstruction and NVS from sparse-view images. We further show that our generalizable pipeline can benefit from preexisting foundational multi-view deep visual features. The resulting model attains the state-of-the-art results on the DTU sparse 3D reconstruction benchmark in terms of Chamfer distance to ground-truth, as-well as state-of-the-art NVS. It also demonstrates strong generalization on the BlendedMVS and Tanks and Temples datasets. We note that our model outperforms the prior state-of-the-art in feed-forward sparse view reconstruction based on volume rendering of implicit representations, while offering an almost 2 orders of magnitude higher inference speed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
