基于条件风格码扩散的可控三维人脸生成
Controllable 3D Face Generation with Conditional Style Code Diffusion
AI总结：
本文提出 TEx-Face 方法，通过三维 GAN 反演、条件风格码扩散和三维人脸解码三个组件，实现基于文本与表情条件的高效可控逼真三维人脸生成。
摘要：
从给定条件生成逼真的三维人脸是一项具有挑战性的任务。现有方法通常依赖耗时的一对一优化方法，这对于建模相同分布内容（例如人脸）而言效率不高。此外，理想的可控三维人脸生成模型应同时考虑面部属性和表情。因此，我们提出了一种名为 TEx-Face（TExt & Expression-to-Face，文本与表情到人脸）的新方法，通过将任务划分为三个组件来应对这些挑战，即三维 GAN 反演、条件风格码扩散和三维人脸解码。对于三维 GAN 反演，我们引入了两种方法，旨在增强风格码的表示能力并缓解三维不一致性问题。此外，我们设计了一个风格码去噪器，以将多个条件融入风格码中，并提出了一种数据增强策略，以解决成对的视觉-语言数据不足的问题。在 FFHQ、CelebA-HQ 和 CelebA-Dialog 上进行的大量实验表明，我们的 TEx-Face 在实现高效且可控的逼真三维人脸生成方面表现出良好的性能。代码将在 https://github.com/sxl142/TEx-Face 上提供。
英文摘要：
Generating photorealistic 3D faces from given conditions is a challenging task. Existing methods often rely on time-consuming one-by-one optimization approaches, which are not efficient for modeling the same distribution content, e.g., faces. Additionally, an ideal controllable 3D face generation model should consider both facial attributes and expressions. Thus we propose a novel approach called TEx-Face(TExt & Expression-to-Face) that addresses these challenges by dividing the task into three components, i.e., 3D GAN Inversion, Conditional Style Code Diffusion, and 3D Face Decoding. For 3D GAN inversion, we introduce two methods which aim to enhance the representation of style codes and alleviate 3D inconsistencies. Furthermore, we design a style code denoiser to incorporate multiple conditions into the style code and propose a data augmentation strategy to address the issue of insufficient paired visual-language data. Extensive experiments conducted on FFHQ, CelebA-HQ, and CelebA-Dialog demonstrate the promising performance of our TEx-Face in achieving the efficient and controllable generation of photorealistic 3D faces. The code will be available at https://github.com/sxl142/TEx-Face.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
