CompoNeRF：具有可编辑3D场景布局的文本引导多物体重组NeRF
CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout
AI总结：
提出CompoNeRF框架，通过集成可编辑3D场景布局与双层文本引导机制，解决多物体场景生成中的引导崩溃问题，实现高保真、可编辑重组的多物体3D场景生成。
摘要：
文本到3D形式在为AR/VR创建可编辑3D场景中起着至关重要的作用。近期进展表明，将神经辐射场（NeRFs）与预训练扩散模型结合用于文本到3D物体生成具有前景。然而，一个长期挑战是其准确解析和重组一致多物体环境的能力不足。具体而言，这些模型难以准确表示多物体文本提示的数量和风格，常导致渲染保真度下降，无法匹配语义复杂性。此外，将这些元素融合为连贯3D场景是一项重大挑战，源于扩散模型固有的通用分布。为解决“引导崩溃”问题并进一步增强场景一致性，我们提出一种新框架CompoNeRF，通过将可编辑3D场景布局与物体特定和场景级引导机制集成。它首先将复杂文本解释为填充多个NeRF的布局，每个NeRF配以相应子文本提示以精确描绘物体。接着，定制组合模块无缝融合这些NeRF，促进一致性，而双层文本引导减少歧义并提升准确性。值得注意的是，我们的组合设计允许分解。这使得基于编辑布局或文本提示进行灵活场景编辑和重组为新场景。利用开源Stable Diffusion模型，CompoNeRF生成高保真多物体场景。显著地，本框架在多视角CLIP分数指标上实现高达54%的提升。用户研究表明，该方法显著改善了多物体场景生成的语义准确性、多视角一致性和个体可识别性。
英文摘要：
Text-to-3D form plays a crucial role in creating editable 3D scenes for AR/VR. Recent advances have shown promise in merging neural radiance fields (NeRFs) with pre-trained diffusion models for text-to-3D object generation. However, one enduring challenge is their inadequate capability to accurately parse and regenerate consistent multi-object environments. Specifically, these models encounter difficulties in accurately representing quantity and style prompted by multi-object texts, often resulting in a collapse of the rendering fidelity that fails to match the semantic intricacies. Moreover, amalgamating these elements into a coherent 3D scene is a substantial challenge, stemming from generic distribution inherent in diffusion models. To tackle the issue of 'guidance collapse' and further enhance scene consistency, we propose a novel framework, dubbed CompoNeRF, by integrating an editable 3D scene layout with object-specific and scene-wide guidance mechanisms. It initiates by interpreting a complex text into the layout populated with multiple NeRFs, each paired with a corresponding subtext prompt for precise object depiction. Next, a tailored composition module seamlessly blends these NeRFs, promoting consistency, while the dual-level text guidance reduces ambiguity and boosts accuracy. Noticeably, our composition design permits decomposition. This enables flexible scene editing and recomposition into new scenes based on the edited layout or text prompts. Utilizing the open-source Stable Diffusion model, CompoNeRF generates multi-object scenes with high fidelity. Remarkably, our framework achieves up to a \textbf{54\%} improvement by the multi-view CLIP score metric. Our user study indicates that our method has significantly improved semantic accuracy, multi-view consistency, and individual recognizability for multi-object scene generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
