DiffusionGAN3D：结合3D GAN与扩散先验提升文本引导的3D生成与域适应
DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priors
AI总结：
针对文本引导3D人像生成与域适应存在的灵活性差、不稳定等问题，提出结合3D GAN与扩散先验的DiffusionGAN3D框架，通过多模块设计提升生成质量与效率。
摘要：
文本引导的3D感知人像域适应与生成在诸多领域有广泛应用。然而，由于训练数据匮乏，且处理几何与外观的高度多样性存在挑战，现有针对这些任务的方法存在灵活性不足、不稳定、保真度低等问题。本文提出一种新型框架DiffusionGAN3D，通过结合3D GAN与扩散先验，提升文本引导的3D域适应与生成效果。具体而言，我们整合了预训练3D生成模型（如EG3D）与文本到图像扩散模型：前者为从文本生成稳定高质量的虚拟形象提供坚实基础，扩散模型则反过来提供强大先验，以富含信息的方向引导3D生成器微调，实现灵活高效的文本引导域适应。为增强域适应的多样性与文本到虚拟形象的生成能力，我们分别引入了相对距离损失与特定案例可学习三平面。此外，我们设计了渐进式纹理细化模块，以提升上述两项任务的纹理质量。大量实验表明，该框架在域适应与文本到虚拟形象任务中均取得优异结果，在生成质量与效率上优于现有方法。项目主页为：https://younglbw.github.io/DiffusionGAN3D-homepage/。
英文摘要：
Text-guided domain adaptation and generation of 3D-aware portraits find many applications in various fields. However, due to the lack of training data and the challenges in handling the high variety of geometry and appearance, the existing methods for these tasks suffer from issues like inflexibility, instability, and low fidelity. In this paper, we propose a novel framework DiffusionGAN3D, which boosts text-guided 3D domain adaptation and generation by combining 3D GANs and diffusion priors. Specifically, we integrate the pre-trained 3D generative models (e.g., EG3D) and text-to-image diffusion models. The former provides a strong foundation for stable and high-quality avatar generation from text. And the diffusion models in turn offer powerful priors and guide the 3D generator finetuning with informative direction to achieve flexible and efficient text-guided domain adaptation. To enhance the diversity in domain adaptation and the generation capability in text-to-avatar, we introduce the relative distance loss and case-specific learnable triplane respectively. Besides, we design a progressive texture refinement module to improve the texture quality for both tasks above. Extensive experiments demonstrate that the proposed framework achieves excellent results in both domain adaptation and text-to-avatar tasks, outperforming existing methods in terms of generation quality and efficiency. The project homepage is at https://younglbw.github.io/DiffusionGAN3D-homepage/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
