FreeScene：面向自由提示的三维场景合成的混合图扩散
FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Prompts
AI总结：
FreeScene 提出混合图扩散 Transformer（MG-DiT），将自由文本和参考图像统一为图表示，在单一模型中实现文本到场景、图到场景和重排，兼顾生成质量与可控性。
摘要：
可控性在三维室内场景合成的实际应用中起着至关重要的作用。现有工作要么允许粗略的基于语言的控制，这种方式方便但缺乏细粒度的场景定制，要么采用基于图的控制，这种方式提供更好的可控性，但需要大量知识来完成繁琐的图设计过程。为了解决这些挑战，我们提出了 FreeScene，一个用户友好的框架，能够实现方便且有效的室内场景合成控制。具体来说，FreeScene 支持自由形式的用户输入，包括文本描述和/或参考图像，允许用户表达多样化的设计意图。用户输入由基于 VLM 的 Graph Designer 充分分析并整合为图表示。然后我们提出了 MG-DiT，一种混合图扩散 Transformer，它执行图感知去噪以增强场景生成。我们的 MG-DiT 不仅擅长保持图结构，而且对多种任务具有广泛的适用性，包括但不限于文本到场景、图到场景和重排，所有这些都在单一模型中完成。大量实验表明，FreeScene 提供了一种高效且用户友好的解决方案，统一了基于文本和基于图的场景合成，在一系列应用中在生成质量和可控性方面均优于最先进的方法。
英文摘要：
Controllability plays a crucial role in the practical applications of 3D indoor scene synthesis. Existing works either allow rough language-based control, that is convenient but lacks fine-grained scene customization, or employ graph based control, which offers better controllability but demands considerable knowledge for the cumbersome graph design process. To address these challenges, we present FreeScene, a user-friendly framework that enables both convenient and effective control for indoor scene synthesis.Specifically, FreeScene supports free-form user inputs including text description and/or reference images, allowing users to express versatile design intentions. The user inputs are adequately analyzed and integrated into a graph representation by a VLM-based Graph Designer. We then propose MG-DiT, a Mixed Graph Diffusion Transformer, which performs graph-aware denoising to enhance scene generation. Our MG-DiT not only excels at preserving graph structure but also offers broad applicability to various tasks, including, but not limited to, text-to-scene, graph-to-scene, and rearrangement, all within a single model. Extensive experiments demonstrate that FreeScene provides an efficient and user-friendly solution that unifies text-based and graph based scene synthesis, outperforming state-of-the-art methods in terms of both generation quality and controllability in a range of applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
