AvatarMMC：基于多模态条件的三维头部虚拟形象生成与编辑
AvatarMMC: 3D Head Avatar Generation and Editing with Multi-Modal Conditioning
AI总结：
提出AvatarMMC，结合3D GAN与潜在扩散模型，实现多模态条件控制的三维头部虚拟形象生成与编辑，性能优于纯GAN方法。
摘要：
我们提出了一种基于三维生成对抗网络（3D GAN）和潜在扩散模型（LDM）的多模态条件三维头部虚拟形象生成与编辑方法。三维生成对抗网络能够在给定单一条件或无条件下生成高质量的头部虚拟形象，然而，生成符合不同模态多个条件的样本具有挑战性。另一方面，潜在扩散模型擅长学习复杂的条件分布。为此，我们提出利用潜在扩散模型的条件能力，实现对预训练三维生成对抗网络潜在空间的多模态控制。我们的方法能够在给定混合控制信号（如RGB输入、分割掩码和全局属性）的情况下生成和编辑三维头部虚拟形象，从而在全局和局部上更好地控制合成虚拟形象的生成与编辑。实验表明，我们提出的方法在生成和编辑任务上，无论在定性还是定量方面均优于仅基于生成对抗网络的方法。据我们所知，我们的方法是首个将多模态条件引入三维虚拟形象生成与编辑的方法。项目页面：avatarmmc-sig24.github.io
英文摘要：
We introduce an approach for 3D head avatar generation and editing with multi-modal conditioning based on a 3D Generative Adversarial Network (GAN) and a Latent Diffusion Model (LDM). 3D GANs can generate high-quality head avatars given a single or no condition. However, it is challenging to generate samples that adhere to multiple conditions of different modalities. On the other hand, LDMs excel at learning complex conditional distributions. To this end, we propose to exploit the conditioning capabilities of LDMs to enable multi-modal control over the latent space of a pre-trained 3D GAN. Our method can generate and edit 3D head avatars given a mixture of control signals such as RGB input, segmentation masks, and global attributes. This provides better control over the generation and editing of synthetic avatars both globally and locally. Experiments show that our proposed approach outperforms a solely GAN-based approach both qualitatively and quantitatively on generation and editing tasks. To the best of our knowledge, our approach is the first to introduce multi-modal conditioning to 3D avatar generation and editing. \\href{avatarmmc-sig24.github.io}{Project Page}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
