你在2D中看到得越多，在3D中感知得就越多
The More You See in 2D, the More You Perceive in 3D
AI总结：
本文提出SAP3D，通过测试时微调视角条件扩散模型并联合估计相机姿态，从任意数量无姿态图像实现随视角增多而提升的3D重建与新视角合成。
摘要：
人类能够根据过往经验从物体的2D图像推断3D结构，并随着看到更多图像而提升对3D的理解。受此行为启发，我们提出SAP3D，这是一个利用任意数量无姿态图像进行3D重建和新视角合成的系统。给定一个物体的若干无姿态图像，我们在测试时微调预训练的视角条件扩散模型，同时估计这些图像的相机姿态。随后，适配后的扩散模型和所得相机姿态被用作实例特定先验，以进行3D重建和新视角合成。我们表明，随着输入图像数量增加，本方法的性能不断提升，弥合了基于优化、无先验的3D重建方法与基于扩散的单图像到3D方法之间的差距。我们在真实图像以及标准合成基准上展示了该系统。消融研究证实，这种适配行为对于更准确的3D理解至关重要。
英文摘要：
Humans can infer 3D structure from 2D images of an object based on past experience and improve their 3D understanding as they see more images. Inspired by this behavior, we introduce SAP3D, a system for 3D reconstruction and novel view synthesis from an arbitrary number of unposed images. Given a few unposed images of an object, we adapt a pre-trained view-conditioned diffusion model together with the camera poses of the images via test-time fine-tuning. The adapted diffusion model and the obtained camera poses are then utilized as instance-specific priors for 3D reconstruction and novel view synthesis. We show that as the number of input images increases, the performance of our approach improves, bridging the gap between optimization-based prior-less 3D reconstruction methods and single-image-to-3D diffusion-based methods. We demonstrate our system on real images as well as standard synthetic benchmarks. Our ablation studies confirm that this adaption behavior is key for more accurate 3D understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
