GeoTransfer：基于迁移学习的可泛化少样本多视角重建
GeoTransfer : Generalizable Few-Shot Multi-View Reconstruction via Transfer Learning
AI总结：
本文提出GeoTransfer方法，通过将预训练可泛化NeRF特征快速迁移至隐式占用网络，实现少样本稀疏三维重建，大幅缩短训练时间，并在DTU和Blended MVS数据集上取得领先精度。
摘要：
本文提出了一种新颖的稀疏三维重建方法，通过利用神经辐射场（NeRFs）的表达能力并快速迁移其特征来学习精确的占用场。现有的稀疏输入三维重建方法在捕捉复杂几何细节方面仍然存在困难，并且在处理被遮挡区域时可能受到限制。另一方面，NeRFs擅长建模复杂场景，但无法提供提取有意义几何信息的手段。我们提出的方法通过迁移NeRF特征中编码的信息来推导精确的占用场表示，从而兼具两者的优势。我们利用一个预训练的、可泛化的最先进NeRF网络来捕捉详细的场景辐射信息，并快速将这些知识迁移到可泛化的隐式占用网络的训练中。这一过程有助于利用编码在可泛化NeRF先验中的场景几何知识，并对其进行精化以学习占用场，从而促进对三维空间更精确的可泛化表示。该迁移学习方法使训练时间大幅减少，减少了几个数量级（即从数天缩短至3.5小时），无需从头训练可泛化的稀疏表面重建方法。此外，我们引入了一种针对体渲染权重的全新损失函数，有助于学习精确的占用场，同时引入一种法线损失，有助于占用场的全局平滑。我们在DTU数据集上评估了我们的方法，并在重建精度方面展示了最先进的性能，尤其是在稀疏输入数据和被遮挡区域等具有挑战性的场景中。我们进一步通过在Blended MVS数据集上展示无需任何重新训练的定性结果，证明了我们方法的泛化能力。
英文摘要：
This paper presents a novel approach for sparse 3D reconstruction by leveraging the expressive power of Neural Radiance Fields (NeRFs) and fast transfer of their features to learn accurate occupancy fields. Existing 3D reconstruction methods from sparse inputs still struggle with capturing intricate geometric details and can suffer from limitations in handling occluded regions. On the other hand, NeRFs excel in modeling complex scenes but do not offer means to extract meaningful geometry. Our proposed method offers the best of both worlds by transferring the information encoded in NeRF features to derive an accurate occupancy field representation. We utilize a pre-trained, generalizable state-of-the-art NeRF network to capture detailed scene radiance information, and rapidly transfer this knowledge to train a generalizable implicit occupancy network. This process helps in leveraging the knowledge of the scene geometry encoded in the generalizable NeRF prior and refining it to learn occupancy fields, facilitating a more precise generalizable representation of 3D space. The transfer learning approach leads to a dramatic reduction in training time, by orders of magnitude (i.e. from several days to 3.5 hrs), obviating the need to train generalizable sparse surface reconstruction methods from scratch. Additionally, we introduce a novel loss on volumetric rendering weights that helps in the learning of accurate occupancy fields, along with a normal loss that helps in global smoothing of the occupancy fields. We evaluate our approach on the DTU dataset and demonstrate state-of-the-art performance in terms of reconstruction accuracy, especially in challenging scenarios with sparse input data and occluded regions. We furthermore demonstrate the generalization capabilities of our method by showing qualitative results on the Blended MVS dataset without any retraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
