GALA3D:通过布局引导的生成式高斯泼溅实现文本到3D复杂场景生成
GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting
AI总结:
本文提出GALA3D框架,利用LLM布局、布局引导3D高斯表示与条件扩散组合优化,实现高保真、可编辑的文本到复杂3D场景生成。
摘要:
我们提出了GALA3D,即具有布局引导控制的生成式3D高斯(generative 3D GAussians with LAyout-guided control),用于有效的组合式文本到3D生成。我们首先利用大型语言模型(LLM)生成初始布局,并引入一种布局引导的3D高斯表示,以带有自适应几何约束的方式进行3D内容生成。随后,我们提出一种结合条件扩散的实例—场景组合优化机制,协同生成在多个对象之间具有一致几何、纹理、尺度以及准确交互关系的逼真3D场景,同时调整从LLM提取的粗略布局先验,使其与生成的场景对齐。实验表明,GALA3D是一个用户友好的端到端框架,可实现最先进的场景级3D内容生成和可控编辑,同时确保场景中对象级实体的高保真度。源代码和模型将在gala3d.github.io提供。
英文摘要:
We present GALA3D, generative 3D GAussians with LAyout-guided control, for effective compositional text-to-3D generation. We first utilize large language models (LLMs) to generate the initial layout and introduce a layout-guided 3D Gaussian representation for 3D content generation with adaptive geometric constraints. We then propose an instance-scene compositional optimization mechanism with conditioned diffusion to collaboratively generate realistic 3D scenes with consistent geometry, texture, scale, and accurate interactions among multiple objects while simultaneously adjusting the coarse layout priors extracted from the LLMs to align with the generated scene. Experiments show that GALA3D is a user-friendly, end-to-end framework for state-of-the-art scene-level 3D content generation and controllable editing while ensuring the high fidelity of object-level entities within the scene. The source codes and models will be available at gala3d.github.io.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。