无需位姿，不是问题：从稀疏未标定图像中生成极其简单的3D高斯泼溅
No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images
AI总结：
本研究提出NoPoSplat前馈模型，通过锚定单视图局部坐标为规范空间、结合相机内参token嵌入，仅用光度损失训练即可从稀疏未标定图像实时重建3D高斯，在新视角合成和位姿估计任务上均优于现有方法。
摘要：
我们提出了NoPoSplat，这是一种前馈模型，能够从**未标定**的稀疏多视图图像中重建以3D高斯为参数的3D场景。我们的模型仅使用光度损失进行训练，在推理阶段可实现实时3D高斯重建。为了消除重建过程中对精确位姿输入的需求，我们将一个输入视图的局部相机坐标锚定为规范空间，并训练网络在该空间内预测所有视图的高斯基元。这种方法无需将高斯基元从局部坐标转换到全局坐标系，从而避免了与逐帧高斯和位姿估计相关的误差。为解决尺度模糊问题，我们设计并比较了多种内参嵌入方法，最终选择将相机内参转换为token嵌入，并将其与图像token拼接作为模型输入，从而实现准确的场景尺度预测。我们将重建的3D高斯用于新视角合成和位姿估计任务，并提出了一种两阶段由粗到精的流程以实现精确的位姿估计。实验结果表明，我们的无位姿方法在新视角合成质量上优于需要位姿的方法，尤其是在输入图像重叠有限的场景中。在位姿估计方面，我们的方法在没有真实深度或显式匹配损失的情况下训练，性能显著优于当前最优方法，提升幅度很大。这项工作在无位姿可泛化3D重建方面取得了重大进展，并证明了其在现实场景中的适用性。代码和训练好的模型可在https://noposplat.github.io/ 获取。
英文摘要：
We introduce NoPoSplat, a feed-forward model capable of reconstructing 3D scenes parameterized by 3D Gaussians from \textit{unposed} sparse multi-view images. Our model, trained exclusively with photometric loss, achieves real-time 3D Gaussian reconstruction during inference. To eliminate the need for accurate pose input during reconstruction, we anchor one input view's local camera coordinates as the canonical space and train the network to predict Gaussian primitives for all views within this space. This approach obviates the need to transform Gaussian primitives from local coordinates into a global coordinate system, thus avoiding errors associated with per-frame Gaussians and pose estimation. To resolve scale ambiguity, we design and compare various intrinsic embedding methods, ultimately opting to convert camera intrinsics into a token embedding and concatenate it with image tokens as input to the model, enabling accurate scene scale prediction. We utilize the reconstructed 3D Gaussians for novel view synthesis and pose estimation tasks and propose a two-stage coarse-to-fine pipeline for accurate pose estimation. Experimental results demonstrate that our pose-free approach can achieve superior novel view synthesis quality compared to pose-required methods, particularly in scenarios with limited input image overlap. For pose estimation, our method, trained without ground truth depth or explicit matching loss, significantly outperforms the state-of-the-art methods with substantial improvements. This work makes significant advances in pose-free generalizable 3D reconstruction and demonstrates its applicability to real-world scenarios. Code and trained models are available at https://noposplat.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
