DITTO-NeRF：基于扩散的迭代文本到全向3D模型
DITTO-NeRF: Diffusion-based Iterative Text To Omni-directional 3D Model
AI总结：
提出DITTO-NeRF流水线，通过修复潜在扩散模型迭代重建3D NeRF，利用渐进式重建方案在保真度和多样性上超越现有方法且训练更快。
摘要：
对高质量3D内容创作日益增长的需求，推动了从单张图像和/或文本提示自动创建3D对象模型方法的发展。然而，使用最先进的image-to-3D方法重建的3D对象，仍然表现出与给定图像的低对应性以及低多视图一致性。最近最先进的text-to-3D方法也受到限制，每个提示生成的3D样本多样性低且合成时间长。为了应对这些挑战，我们提出了DITTO-NeRF，一种从文本提示或单张图像生成高质量3D NeRF模型的新颖流水线。我们的DITTO-NeRF包括：使用前视图的给定或文本生成的2D图像，为有限的内部边界（IB）角度构建高质量的部分3D对象，然后使用修复潜在扩散模型迭代重建剩余的3D NeRF。我们在DITTO-NeRF中提出了渐进式3D对象重建方案，涉及尺度（低分辨率到高分辨率）、角度（最初为IB角度，随后为外部边界（OB））和掩码（对象到背景边界），以便IB上的高质量信息可以传播到OB中。我们的DITTO-NeRF在保真度和多样性上定性定量地优于最先进的方法，并且与DreamFusion和NeuralLift-360等先前的image/text-to-3D方法相比，训练时间快得多。
英文摘要：
The increasing demand for high-quality 3D content creation has motivated the development of automated methods for creating 3D object models from a single image and/or from a text prompt. However, the reconstructed 3D objects using state-of-the-art image-to-3D methods still exhibit low correspondence to the given image and low multi-view consistency. Recent state-of-the-art text-to-3D methods are also limited, yielding 3D samples with low diversity per prompt with long synthesis time. To address these challenges, we propose DITTO-NeRF, a novel pipeline to generate a high-quality 3D NeRF model from a text prompt or a single image. Our DITTO-NeRF consists of constructing high-quality partial 3D object for limited in-boundary (IB) angles using the given or text-generated 2D image from the frontal view and then iteratively reconstructing the remaining 3D NeRF using inpainting latent diffusion model. We propose progressive 3D object reconstruction schemes in terms of scales (low to high resolution), angles (IB angles initially to outer-boundary (OB) later), and masks (object to background boundary) in our DITTO-NeRF so that high-quality information on IB can be propagated into OB. Our DITTO-NeRF outperforms state-of-the-art methods in terms of fidelity and diversity qualitatively and quantitatively with much faster training times than prior arts on image/text-to-3D such as DreamFusion, and NeuralLift-360.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
