arXivDaily arXiv每日学术速递 周一至周五更新
​通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2502.10675cs.CV

利用预训练大语言模型进行层次结构化的开放词汇室内场景合成

Hierarchically-Structured Open-Vocabulary Indoor Scene Synthesis with Pre-trained Large Language Model

School of Software, Shandong University（山东大学软件学院）；School of Computer Science, National University of Defense Technology（国防科技大学计算机学院）；Shandong Research Institute of Industrial Technology（山东省产业技术研究院）

Weilin Sun, Xinran Li, Manyi Li, Kai Xu, Xiangxu Meng, Lei Meng

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对开放词汇室内场景合成中LLM输出难以转化为可行布局的问题，提出层次化场景描述、层次感知位置推断网络与分而治之布局优化，提升布局合理性及需求一致性。

摘要：

室内场景合成旨在自动生成合理、逼真且多样的3D室内场景，尤其是在给定任意用户需求的情况下。近年来，预训练大语言模型（LLM）所具有的良好泛化能力为开放词汇室内场景合成提供了帮助。然而，挑战在于如何将LLM生成的输出转换为合理且物理可行的场景布局。本文提出利用LLM生成层次结构化的场景描述，随后计算场景布局。具体而言，我们训练一个层次感知网络来推断物体之间细粒度的相对位置，并设计一种分而治之的优化方法来求解场景布局。使用层次结构化场景表示的优势有两方面：首先，层次结构为物体排列提供了粗略的定位基础，从而缓解密集关系下相互矛盾的摆放问题，并增强网络推断细粒度摆放的泛化能力；其次，该结构自然支持分而治之的优化，即先布置子场景，再布置整个场景，从而更有效地求解可行布局。我们通过定性和定量评估开展了大量比较实验与消融研究，以验证采用层次结构化场景表示的关键设计的有效性。我们的方法能够生成更合理的场景布局，同时更好地与用户需求及LLM描述保持一致。我们还展示了开放词汇场景合成和交互式场景设计结果，以体现本方法在应用中的优势。

英文摘要：

Indoor scene synthesis aims to automatically produce plausible, realistic and diverse 3D indoor scenes, especially given arbitrary user requirements. Recently, the promising generalization ability of pre-trained large language models (LLM) assist in open-vocabulary indoor scene synthesis. However, the challenge lies in converting the LLM-generated outputs into reasonable and physically feasible scene layouts. In this paper, we propose to generate hierarchically structured scene descriptions with LLM and then compute the scene layouts. Specifically, we train a hierarchy-aware network to infer the fine-grained relative positions between objects and design a divide-and-conquer optimization to solve for scene layouts. The advantages of using hierarchically structured scene representation are two-fold. First, the hierarchical structure provides a rough grounding for object arrangement, which alleviates contradictory placements with dense relations and enhances the generalization ability of the network to infer fine-grained placements. Second, it naturally supports the divide-and-conquer optimization, by first arranging the sub-scenes and then the entire scene, to more effectively solve for a feasible layout. We conduct extensive comparison experiments and ablation studies with both qualitative and quantitative evaluations to validate the effectiveness of our key designs with the hierarchically structured scene representation. Our approach can generate more reasonable scene layouts while better aligned with the user requirements and LLM descriptions. We also present open-vocabulary scene synthesis and interactive scene design results to show the strength of our approach in the applications.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑