UVRM：一种可扩展的无位姿视频三维重建模型
UVRM: A Scalable 3D Reconstruction Model from Unposed Videos
AI总结：
本文提出UVRM模型，通过Transformer聚合视频帧到位姿不变潜在空间并解码为三平面表示，结合SDS与综合分析生成伪新视角，首次实现无需位姿标注的单目视频三维重建。
摘要：
大型重建模型（LRMs）近年来已成为创建三维基础模型的一种流行方法。使用二维视觉数据训练三维重建模型，传统上需要预先知道训练样本的相机位姿，这一过程既耗时又容易出错。因此，三维重建训练一直被限制在合成三维数据集或带有位姿标注的小规模数据集上。在本研究中，我们探讨了利用各种物体的无位姿视频数据进行三维重建的可行性。我们提出了UVRM，这是一种新颖的三维重建模型，能够在单目视频上进行训练和评估，而无需任何位姿信息。UVRM使用Transformer网络将视频帧隐式聚合到一个位姿不变的潜在特征空间中，然后将其解码为三平面三维表示。为了在训练过程中免除对真实位姿标注的需求，UVRM结合使用了分数蒸馏采样（SDS）方法和综合分析（analysis-by-synthesis）方法，利用预训练的扩散模型逐步合成伪新视角。我们在G-Objaverse和CO3D数据集上，在不依赖位姿信息的情况下，对UVRM的性能进行了定性和定量评估。大量实验表明，UVRM能够有效且高效地从无位姿视频中重建出各种各样的三维物体。
英文摘要：
Large Reconstruction Models (LRMs) have recently become a popular method for creating 3D foundational models. Training 3D reconstruction models with 2D visual data traditionally requires prior knowledge of camera poses for the training samples, a process that is both time-consuming and prone to errors. Consequently, 3D reconstruction training has been confined to either synthetic 3D datasets or small-scale datasets with annotated poses. In this study, we investigate the feasibility of 3D reconstruction using unposed video data of various objects. We introduce UVRM, a novel 3D reconstruction model capable of being trained and evaluated on monocular videos without requiring any information about the pose. UVRM uses a transformer network to implicitly aggregate video frames into a pose-invariant latent feature space, which is then decoded into a tri-plane 3D representation. To obviate the need for ground-truth pose annotations during training, UVRM employs a combination of the score distillation sampling (SDS) method and an analysis-by-synthesis approach, progressively synthesizing pseudo novel-views using a pre-trained diffusion model. We qualitatively and quantitatively evaluate UVRM's performance on the G-Objaverse and CO3D datasets without relying on pose information. Extensive experiments show that UVRM is capable of effectively and efficiently reconstructing a wide range of 3D objects from unposed videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
