面向未定位图像的大规模新视角合成的即时重建
On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images
AI总结：
本文提出一种即时重建方法，通过快速初始位姿估计、直接采样高斯基元以及增量式聚类与GPU卸载，在拍摄完成后立即为大规模未定位图像生成相机位姿和训练好的3DGS，兼顾速度与图像质量。
摘要：
诸如3D高斯泼溅（3DGS）等辐射场方法能够从照片中轻松重建场景，实现自由视点导航。然而，利用运动恢复结构（Structure from Motion）进行位姿估计以及3DGS优化在拍摄完成后仍可能各自需要数分钟到数小时的计算时间。结合3DGS的SLAM方法速度较快，但难以应对宽相机基线和大型场景。我们提出了一种即时（on-the-fly）方法，可在拍摄完成后立即生成相机位姿和训练好的3DGS。我们的方法能够处理有序照片序列的稠密、宽基线拍摄以及大规模场景。为此，我们首先引入快速初始位姿估计，利用学习到的特征和GPU友好的迷你光束法平差（mini bundle adjustment）。随后，我们引入高斯基元位置和形状的直接采样，在需要的位置增量式生成基元，从而显著加速训练。这两个高效步骤实现了位姿和高斯基元的快速且鲁棒的联合优化。我们的增量式方法通过引入可扩展的辐射场构建来处理大规模场景，逐步对3DGS基元进行聚类，将其存储在锚点中，并将其从GPU卸载。聚类后的基元被逐步合并，从而在任意视点下保持所需的3DGS规模。我们在多种数据集上评估了我们的解决方案，并表明我们的方案能够对我们所针对的所有拍摄场景和场景规模进行即时处理，同时在速度、图像质量或两者上保持与仅处理特定拍摄风格或场景规模的其他方法相比的竞争力。
英文摘要：
Radiance field methods such as 3D Gaussian Splatting (3DGS) allow easy reconstruction from photos, enabling free-viewpoint navigation. Nonetheless, pose estimation using Structure from Motion and 3DGS optimization can still each take between minutes and hours of computation after capture is complete. SLAM methods combined with 3DGS are fast but struggle with wide camera baselines and large scenes. We present an on-the-fly method to produce camera poses and a trained 3DGS immediately after capture. Our method can handle dense and wide-baseline captures of ordered photo sequences and large-scale scenes. To do this, we first introduce fast initial pose estimation, exploiting learned features and a GPU-friendly mini bundle adjustment. We then introduce direct sampling of Gaussian primitive positions and shapes, incrementally spawning primitives where required, significantly accelerating training. These two efficient steps allow fast and robust joint optimization of poses and Gaussian primitives. Our incremental approach handles large-scale scenes by introducing scalable radiance field construction, progressively clustering 3DGS primitives, storing them in anchors, and offloading them from the GPU. Clustered primitives are progressively merged, keeping the required scale of 3DGS at any viewpoint. We evaluate our solution on a variety of datasets and show that our solution can provide on-the-fly processing of all the capture scenarios and scene sizes we target while remaining competitive with other methods that only handle specific capture styles or scene sizes in speed, image quality, or both.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
