文本引导的三维人脸合成——从生成到编辑
Text-Guided 3D Face Synthesis -- From Generation to Editing
AI总结：
本文提出统一的文本引导三维人脸生成与编辑框架，通过几何-纹理解耦生成、UV域一致性保持正则化和自引导一致性权重策略，实现高质量且可迭代编辑的定制化三维人脸合成。
摘要：
文本引导的三维人脸合成通过利用文本到图像（T2I）扩散模型已经取得了显著成果。然而，现有大多数工作仅关注直接生成，忽略了编辑，这限制了它们通过迭代调整来合成定制化三维人脸的能力。在本文中，我们提出了一个统一的文本引导框架，涵盖从人脸生成到编辑的全过程。在生成阶段，我们提出了一种几何-纹理解耦生成方法，以缓解由耦合引起的几何细节损失。此外，解耦使我们能够利用生成的几何作为纹理生成的条件，从而获得高度几何-纹理对齐的结果。我们进一步采用微调后的纹理扩散模型，在RGB和YUV空间中提升纹理质量。在编辑阶段，我们首先采用预训练的扩散模型，根据文本更新人脸几何或纹理。为了实现顺序编辑，我们引入了UV域一致性保持正则化，防止对无关人脸属性的无意更改。此外，我们提出了一种自引导一致性权重策略，在保持一致性的同时提高编辑效果。通过全面的实验，我们展示了该方法在人脸合成中的优越性。项目页面：https://faceg2e.github.io/。
英文摘要：
Text-guided 3D face synthesis has achieved remarkable results by leveraging text-to-image (T2I) diffusion models. However, most existing works focus solely on the direct generation, ignoring the editing, restricting them from synthesizing customized 3D faces through iterative adjustments. In this paper, we propose a unified text-guided framework from face generation to editing. In the generation stage, we propose a geometry-texture decoupled generation to mitigate the loss of geometric details caused by coupling. Besides, decoupling enables us to utilize the generated geometry as a condition for texture generation, yielding highly geometry-texture aligned results. We further employ a fine-tuned texture diffusion model to enhance texture quality in both RGB and YUV space. In the editing stage, we first employ a pre-trained diffusion model to update facial geometry or texture based on the texts. To enable sequential editing, we introduce a UV domain consistency preservation regularization, preventing unintentional changes to irrelevant facial attributes. Besides, we propose a self-guided consistency weight strategy to improve editing efficacy while preserving consistency. Through comprehensive experiments, we showcase our method's superiority in face synthesis. Project page: https://faceg2e.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
