Ctrl-Room：基于布局约束的可控文本到三维房间网格生成
Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints
AI总结：
Ctrl-Room提出两阶段方法，通过布局生成和外观生成分离建模，实现从文本提示生成可编辑的高质量三维房间网格，并在Structured3D数据集上优于现有方法。
摘要：
文本驱动的三维室内场景生成对于游戏、电影行业以及增强现实/虚拟现实应用非常有用。然而，现有方法无法忠实地捕捉房间布局，也不允许对房间内的单个物体进行灵活编辑。为了解决这些问题，我们提出了Ctrl-Room，它仅凭文本提示即可生成具有设计师风格布局和高保真纹理的逼真三维房间。此外，Ctrl-Room支持多种交互式编辑操作，例如调整单个家具的大小或移动它们。我们的关键见解是将布局和外观的建模分开。我们提出的方法包括两个阶段：布局生成阶段和外观生成阶段。布局生成阶段训练一个文本条件扩散模型，通过我们的整体场景代码参数化来学习布局分布。接下来，外观生成阶段采用微调后的ControlNet，在三维场景布局和文本提示的引导下生成生动的房间全景图像。因此，我们实现了具有令人信服的布局和生动纹理的高质量三维房间生成。得益于场景代码参数化，我们可以通过掩码引导的编辑模块轻松编辑生成的三维房间模型，而无需昂贵的针对编辑的训练。在Structured3D数据集上进行的大量实验表明，我们的方法在从自然语言提示生成更合理、视角一致且可编辑的三维房间方面优于现有方法。
英文摘要：
Text-driven 3D indoor scene generation is useful for gaming, the film industry, and AR/VR applications. However, existing methods cannot faithfully capture the room layout, nor do they allow flexible editing of individual objects in the room. To address these problems, we present Ctrl-Room, which can generate convincing 3D rooms with designer-style layouts and high-fidelity textures from just a text prompt. Moreover, Ctrl-Room enables versatile interactive editing operations such as resizing or moving individual furniture items. Our key insight is to separate the modeling of layouts and appearance. Our proposed method consists of two stages: a Layout Generation Stage and an Appearance Generation Stage. The Layout Generation Stage trains a text-conditional diffusion model to learn the layout distribution with our holistic scene code parameterization. Next, the Appearance Generation Stage employs a fine-tuned ControlNet to produce a vivid panoramic image of the room guided by the 3D scene layout and text prompt. We thus achieve a high-quality 3D room generation with convincing layouts and lively textures. Benefiting from the scene code parameterization, we can easily edit the generated room model through our mask-guided editing module, without expensive edit-specific training. Extensive experiments on the Structured3D dataset demonstrate that our method outperforms existing methods in producing more reasonable, view-consistent, and editable 3D rooms from natural language prompts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
