PhiP-G：物理引导的文本到3D组合场景生成
PhiP-G: Physics-Guided Text-to-3D Compositional Scene Generation
AI总结：
针对文本到3D组合场景生成中布局不符物理定律、复杂描述捕捉难等问题，提出框架 PhiP-G。该框架结合LLM智能体、多模态2D生成与3D高斯生成构建物理世界模型，显著提升了场景生成质量、物理合理性及效率。
摘要：
文本到3D资产生成在2D扩散先验的监督下取得了显著的优化。然而，在处理组合场景时，现有方法面临几个挑战：1). 未能确保组合场景布局符合物理定律；2). 难以准确捕捉复杂场景描述中所述的资产与关系；3). 利用大型语言模型（LLMs）的布局方法在自主资产生成能力方面存在局限。为避免这些妥协，我们提出了一种用于组合场景生成的新型框架 PhiP-G，该框架将生成技术与基于世界模型的布局引导无缝集成。借助基于LLM的智能体，PhiP-G 分析复杂场景描述以生成场景图，并整合多模态2D生成智能体和3D高斯生成方法来进行目标资产创建。在布局阶段，PhiP-G 采用具有附着能力的物理池和视觉监督智能体，构建用于布局预测与规划的世界模型。大量实验表明，PhiP-G 显著提升了组合场景的生成质量与物理合理性。值得注意的是，PhiP-G 在 CLIP 分数上取得了最先进（SOTA）的性能，在通过 T$^3$Bench 衡量的生成质量上与领先方法持平，并将效率提升了24倍。
英文摘要：
Text-to-3D asset generation has achieved significant optimization under the supervision of 2D diffusion priors. However, when dealing with compositional scenes, existing methods encounter several challenges: 1). failure to ensure that composite scene layouts comply with physical laws; 2). difficulty in accurately capturing the assets and relationships described in complex scene descriptions; 3). limited autonomous asset generation capabilities among layout approaches leveraging large language models (LLMs). To avoid these compromises, we propose a novel framework for compositional scene generation, PhiP-G, which seamlessly integrates generation techniques with layout guidance based on a world model. Leveraging LLM-based agents, PhiP-G analyzes the complex scene description to generate a scene graph, and integrating a multimodal 2D generation agent and a 3D Gaussian generation method for targeted assets creation. For the stage of layout, PhiP-G employs a physical pool with adhesion capabilities and a visual supervision agent, forming a world model for layout prediction and planning. Extensive experiments demonstrate that PhiP-G significantly enhances the generation quality and physical rationality of the compositional scenes. Notably, PhiP-G attains state-of-the-art (SOTA) performance in CLIP scores, achieves parity with the leading methods in generation quality as measured by the T$^3$Bench, and improves efficiency by 24x.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
