InstructScene：基于语义图先验的指令驱动三维室内场景合成
InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior
AI总结：
针对三维室内场景合成中自然语言指令可控性不足的问题，提出InstructScene框架，结合语义图先验与布局解码器，提升生成可控性与保真度，并构建高质量数据集，实验大幅超越现有方法。
摘要：
理解自然语言指令是三维室内场景合成系统的一个迷人特性。现有方法直接建模对象的联合分布，并在场景内隐式表达对象间关系，从而阻碍了生成的可控性。我们提出了InstructScene，一种新颖的生成框架，它整合了语义图先验和布局解码器，以提高三维场景合成的可控性和保真度。所提出的语义图先验联合学习场景外观和布局分布，以零样本方式展现出跨各种下游任务的多功能性。为了促进文本驱动三维场景合成的基准测试，我们利用大型语言和多模态模型，精心策划了一个高质量的“场景-指令”配对数据集。大量实验结果表明，所提出的方法大幅超越了现有的最先进方法。彻底的消融研究证实了关键设计组件的有效性。项目页面：https://chenguolin.github.io/projects/InstructScene。
英文摘要：
Comprehending natural language instructions is a charming property for 3D indoor scene synthesis systems. Existing methods directly model object joint distributions and express object relations implicitly within a scene, thereby hindering the controllability of generation. We introduce InstructScene, a novel generative framework that integrates a semantic graph prior and a layout decoder to improve controllability and fidelity for 3D scene synthesis. The proposed semantic graph prior jointly learns scene appearances and layout distributions, exhibiting versatility across various downstream tasks in a zero-shot manner. To facilitate the benchmarking for text-driven 3D scene synthesis, we curate a high-quality dataset of scene-instruction pairs with large language and multimodal models. Extensive experimental results reveal that the proposed method surpasses existing state-of-the-art approaches by a large margin. Thorough ablation studies confirm the efficacy of crucial design components. Project page: https://chenguolin.github.io/projects/InstructScene.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
