从无到有:通过新视角合成的自监督3D重建
From None to All: Self-Supervised 3D Reconstruction via Novel View Synthesis
AI总结:
提出NAS3R框架,通过自监督学习从无标注图像中联合估计3D几何和相机参数,利用新视角合成进行训练,无需真实标注或预训练先验。
摘要:
本文提出NAS3R,一个自监督前馈框架,无需真实标注和预训练先验,联合学习显式3D几何和相机参数。训练时,NAS3R从无标定和无位姿的上下文视图重建3D高斯,并使用自预测的相机参数渲染目标视图,从而通过2D光度监督实现自监督训练。为确保稳定收敛,NAS3R在共享的Transformer骨干中集成重建和相机预测,并由掩码注意力调控,同时采用基于深度的高斯公式以促进良态优化。该框架与最先进的监督3D重建架构兼容,并可在可用时融入预训练先验或内参信息。大量实验表明,NAS3R优于其他自监督方法,为从无约束数据中进行3D重建建立了一个可扩展且几何感知的范式。代码和模型已在https://ranrhuang.github.io/nas3r/公开。
英文摘要:
In this paper, we introduce NAS3R, a self-supervised feed-forward framework that jointly learns explicit 3D geometry and camera parameters with no ground-truth annotations and no pretrained priors. During training, NAS3R reconstructs 3D Gaussians from uncalibrated and unposed context views and renders target views using its self-predicted camera parameters, enabling self-supervised training from 2D photometric supervision. To ensure stable convergence, NAS3R integrates reconstruction and camera prediction within a shared transformer backbone regulated by masked attention, and adopts a depth-based Gaussian formulation that facilitates well-conditioned optimization. The framework is compatible with state-of-the-art supervised 3D reconstruction architectures and can incorporate pretrained priors or intrinsic information when available. Extensive experiments show that NAS3R achieves superior results to other self-supervised methods, establishing a scalable and geometry-aware paradigm for 3D reconstruction from unconstrained data. Code and models are publicly available at https://ranrhuang.github.io/nas3r/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。