开放宇宙室内场景生成：基于LLM程序合成与未整理物体数据库
Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases
AI总结：
提出利用LLM程序合成与VLM检索未整理网格数据库的室内场景生成系统，无需3D训练数据，在开放与封闭宇宙任务上均超越现有方法。
摘要：
我们提出一个系统，用于响应文本提示生成室内场景。提示不限于固定的场景描述词汇，生成场景中的物体也不限于固定的物体类别集合——我们将此设置称为室内场景生成。与大多数先前关于室内场景生成的工作不同，我们的系统不需要大型现有3D场景训练数据集。相反，它利用预训练大型语言模型（LLMs）中编码的世界知识，在特定领域的布局语言中合成程序，描述物体及其之间的空间关系。执行这样的程序产生一个约束满足问题的规范，系统使用基于梯度的优化方案求解，以生成物体位置和方向。为了生成物体几何，系统从数据库中检索3D网格。与先前使用类别标注、相互对齐网格数据库的工作不同，我们开发了一个使用视觉语言模型（VLMs）从大规模未标注、不一致对齐网格数据库中检索网格的流程。实验评估表明，我们的系统在传统封闭宇宙场景生成任务上优于基于3D数据训练的生成模型；在开放宇宙场景生成上也优于最近的基于LLM的布局生成方法。
英文摘要：
We present a system for generating indoor scenes in response to text prompts. The prompts are not limited to a fixed vocabulary of scene descriptions, and the objects in generated scenes are not restricted to a fixed set of object categories -- we call this setting indoor scene generation. Unlike most prior work on indoor scene generation, our system does not require a large training dataset of existing 3D scenes. Instead, it leverages the world knowledge encoded in pre-trained large language models (LLMs) to synthesize programs in a domain-specific layout language that describe objects and spatial relations between them. Executing such a program produces a specification of a constraint satisfaction problem, which the system solves using a gradient-based optimization scheme to produce object positions and orientations. To produce object geometry, the system retrieves 3D meshes from a database. Unlike prior work which uses databases of category-annotated, mutually-aligned meshes, we develop a pipeline using vision-language models (VLMs) to retrieve meshes from massive databases of un-annotated, inconsistently-aligned meshes. Experimental evaluations show that our system outperforms generative models trained on 3D data for traditional, closed-universe scene generation tasks; it also outperforms a recent LLM-based layout generation method on open-universe scene generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
