Layout2Scene：通过几何与外观扩散先验实现的3D语义布局引导场景生成
Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors
AI总结：
针对文本条件3D场景生成缺乏精确位置控制的问题，本文提出Layout2Scene，利用语义布局并结合两阶段几何与外观扩散先验优化，生成更逼真、可灵活精确编辑的3D场景。
摘要：
得益于2D扩散生成模型的发展，以文本提示为条件的3D场景生成已取得显著进展。然而，3D场景的文本描述本身并不精确，并且在训练过程中缺乏细粒度控制，从而导致生成的场景不合理。作为一种直观且可行的解决方案，3D布局能够精确指定场景中物体的位置。为此，我们提出了一种文本到场景生成方法（即Layout2Scene），该方法使用额外的语义布局作为提示，以注入对3D物体位置的精确控制。具体而言，我们首先引入一种场景混合表示，以解耦物体和背景，并通过预训练的文本到3D模型对其进行初始化。随后，我们提出一种两阶段方案，分别优化初始化场景的几何和外观。为了在几何与外观生成中充分利用2D扩散先验，我们引入了语义引导的几何扩散模型和语义-几何引导的扩散模型，并在场景数据集上对其进行微调。大量实验表明，与最先进的方法相比，我们的方法能够生成更加合理和逼真的场景。此外，生成的场景支持灵活而精确的编辑，从而促进多种下游应用。
英文摘要：
3D scene generation conditioned on text prompts has significantly progressed due to the development of 2D diffusion generation models. However, the textual description of 3D scenes is inherently inaccurate and lacks fine-grained control during training, leading to implausible scene generation. As an intuitive and feasible solution, the 3D layout allows for precise specification of object locations within the scene. To this end, we present a text-to-scene generation method (namely, Layout2Scene) using additional semantic layout as the prompt to inject precise control of 3D object positions. Specifically, we first introduce a scene hybrid representation to decouple objects and backgrounds, which is initialized via a pre-trained text-to-3D model. Then, we propose a two-stage scheme to optimize the geometry and appearance of the initialized scene separately. To fully leverage 2D diffusion priors in geometry and appearance generation, we introduce a semantic-guided geometry diffusion model and a semantic-geometry guided diffusion model which are finetuned on a scene dataset. Extensive experiments demonstrate that our method can generate more plausible and realistic scenes as compared to state-of-the-art approaches. Furthermore, the generated scene allows for flexible yet precise editing, thereby facilitating multiple downstream applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
