Pragmatist：用于从无位姿稀疏视图进行高保真3D重建的多视图条件扩散模型
Pragmatist: Multiview Conditional Diffusion Models for High-Fidelity 3D Reconstruction from Unposed Sparse Views
AI总结：
针对无位姿稀疏视图3D重建细节缺失的问题，提出Pragmatist流水线，通过多视图条件扩散生成完整观测、前馈模型重建并反演优化，提升了重建质量。
摘要：
从稀疏、无位姿的观测中推断3D结构因不受约束的特性而极具挑战性。近期的方法提出以数据驱动的方式直接从无位姿输入预测隐式表示，取得了可观的成果。然而，这些方法未利用几何先验，也无法臆测未见区域的外观，因此难以重建精细的几何与纹理细节。为应对这一挑战，我们的核心思路是将这一不适定问题重新表述为条件新视图合成，旨在从有限的输入视图生成完整观测以助力重建。有了完整观测后，输入视图的位姿可轻松恢复，进而用于优化重建对象。为此，我们提出了一种新型流水线Pragmatist。首先，我们通过多视图条件扩散模型生成对象的完整观测。接着，我们使用前馈大型重建模型得到重建网格。为进一步提升重建质量，我们通过反演所得的3D表示恢复输入视图的位姿，并利用细节丰富的输入视图进一步优化纹理。与以往方法不同，我们的流水线通过高效利用无位姿输入与生成式先验来改进重建，规避了直接求解高度不适定的问题。大量实验表明，我们的方法在多个基准测试中取得了可观的性能。
英文摘要：
Inferring 3D structures from sparse, unposed observations is challenging due to its unconstrained nature. Recent methods propose to predict implicit representations directly from unposed inputs in a data-driven manner, achieving promising results. However, these methods do not utilize geometric priors and cannot hallucinate the appearance of unseen regions, thus making it challenging to reconstruct fine geometric and textural details. To tackle this challenge, our key idea is to reformulate this ill-posed problem as conditional novel view synthesis, aiming to generate complete observations from limited input views to facilitate reconstruction. With complete observations, the poses of the input views can be easily recovered and further used to optimize the reconstructed object. To this end, we propose a novel pipeline Pragmatist. First, we generate a complete observation of the object via a multiview conditional diffusion model. Then, we use a feed-forward large reconstruction model to obtain the reconstructed mesh. To further improve the reconstruction quality, we recover the poses of input views by inverting the obtained 3D representations and further optimize the texture using detailed input views. Unlike previous approaches, our pipeline improves reconstruction by efficiently leveraging unposed inputs and generative priors, circumventing the direct resolution of highly ill-posed problems. Extensive experiments show that our approach achieves promising performance in several benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
