LLplace：基于大语言模型的3D室内场景布局生成与编辑
LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Model
AI总结：
本文提出LLplace，一种基于轻量级微调开源大语言模型Llama3的3D室内场景布局生成与编辑方法，无需空间关系先验和上下文示例，仅凭用户输入即可通过对话实现高质量布局生成及物体增删的动态编辑。
摘要：
设计3D室内布局是一项关键任务，在虚拟现实、室内设计和自动化空间规划中具有重要应用。现有的3D布局设计方法要么依赖利用空间关系先验的扩散模型，要么大量借助专有大语言模型（LLMs）的推理能力，后者需要通过黑盒试验进行大量提示工程和上下文示例设计。这些方法在泛化能力和动态场景编辑方面往往存在局限。在本文中，我们提出了LLplace，一种基于轻量级微调的开源大语言模型Llama3的新型3D室内场景布局设计器。LLplace绕开了对空间关系先验和上下文示例的需求，仅根据用户指定的房间类型和所需物体输入，即可实现高效且可信的房间布局生成。我们基于3D-Front数据集构建了一个新的对话数据集，扩展了原始数据量，并加入了用于添加和移除物体的对话数据。该数据集能够增强大语言模型的空间理解能力。此外，通过对话，LLplace激活了大语言模型理解3D布局并执行动态场景编辑的能力，从而支持物体的添加和移除。我们的方法表明，LLplace能够以交互方式有效生成和编辑3D室内布局，并在提供高质量3D设计方案方面优于现有方法。代码和数据集将公开发布。
英文摘要：
Designing 3D indoor layouts is a crucial task with significant applications in virtual reality, interior design, and automated space planning. Existing methods for 3D layout design either rely on diffusion models, which utilize spatial relationship priors, or heavily leverage the inferential capabilities of proprietary Large Language Models (LLMs), which require extensive prompt engineering and in-context exemplars via black-box trials. These methods often face limitations in generalization and dynamic scene editing. In this paper, we introduce LLplace, a novel 3D indoor scene layout designer based on lightweight fine-tuned open-source LLM Llama3. LLplace circumvents the need for spatial relationship priors and in-context exemplars, enabling efficient and credible room layout generation based solely on user inputs specifying the room type and desired objects. We curated a new dialogue dataset based on the 3D-Front dataset, expanding the original data volume and incorporating dialogue data for adding and removing objects. This dataset can enhance the LLM's spatial understanding. Furthermore, through dialogue, LLplace activates the LLM's capability to understand 3D layouts and perform dynamic scene editing, enabling the addition and removal of objects. Our approach demonstrates that LLplace can effectively generate and edit 3D indoor layouts interactively and outperform existing methods in delivering high-quality 3D design solutions. Code and dataset will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
