基于预训练多视图扩散模型的具组合性与多样性的文本到3D生成
Grounded Compositional and Diverse Text-to-3D with Pretrained Multi-View Diffusion Model
AI总结：
提出Grounded-Dreamer两阶段方法，利用预训练多视图扩散模型和注意力重聚焦机制，解决文本到3D生成中组合提示遗漏问题，实现高保真且多样化的3D资产生成。
摘要：
本文提出了一种名为Grounded-Dreamer的有效两阶段方法，通过使用预训练的多视图扩散模型，生成能够准确遵循复杂组合文本提示的3D资产，同时实现高保真度。诸如MVDream的多视图扩散模型已被证明可以使用分数蒸馏采样（SDS）生成高保真3D资产。然而，如果简单直接地应用，这些方法往往无法理解组合文本提示，并且可能经常完全遗漏某些主体或部分。为了解决这个问题，我们首先提倡利用文本引导的4视图图像作为文本到3D流程的瓶颈。然后，我们引入了注意力重聚焦机制以鼓励文本对齐的4视图图像生成，而无需重新训练多视图扩散模型或精心构建高质量的组合3D数据集。我们进一步提出了一种混合优化策略，以促进SDS损失与稀疏RGB参考图像之间的协同作用。我们的方法在生成组合3D资产方面始终优于先前的最先进（SOTA）方法，在质量和准确性上均表现出色，并能基于相同的文本提示实现多样化的3D生成。
英文摘要：
In this paper, we propose an effective two-stage approach named Grounded-Dreamer to generate 3D assets that can accurately follow complex, compositional text prompts while achieving high fidelity by using a pre-trained multi-view diffusion model. Multi-view diffusion models, such as MVDream, have shown to generate high-fidelity 3D assets using score distillation sampling (SDS). However, applied naively, these methods often fail to comprehend compositional text prompts, and may often entirely omit certain subjects or parts. To address this issue, we first advocate leveraging text-guided 4-view images as the bottleneck in the text-to-3D pipeline. We then introduce an attention refocusing mechanism to encourage text-aligned 4-view image generation, without the necessity to re-train the multi-view diffusion model or craft a high-quality compositional 3D dataset. We further propose a hybrid optimization strategy to encourage synergy between the SDS loss and the sparse RGB reference images. Our method consistently outperforms previous state-of-the-art (SOTA) methods in generating compositional 3D assets, excelling in both quality and accuracy, and enabling diverse 3D from the same text prompt.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
