高保真3D人脸数字人合成与可控细粒度表情生成
High-Fidelity 3D Facial Avatar Synthesis with Controllable Fine-Grained Expressions
AI总结：
本文提出一种同时优化预训练3D感知GAN模型的潜在代码和驱动3DMM模型的表情代码的方法，通过双映射模块和文本引导优化实现高保真3D人脸合成与细粒度表情控制。
摘要：
面部表情编辑方法主要分为基于2D和3D的两种类型。前者缺乏3D面部建模能力，难以有效编辑3D因素。后者在使用单视图2D人脸图像生成高质量且视角一致的渲染效果上表现优异。尽管这些方法已成功使用可动画模型控制面部表情，但仍存在无法精确控制细粒度表情的局限。为解决此问题，本文提出一种新方法，同时优化预训练3D-Aware GAN模型的潜在代码用于纹理编辑和驱动3DMM模型的表情代码用于网格编辑。具体而言，我们引入了包含纹理映射器和情绪映射器的双映射模块，分别学习给定潜在代码的纹理转换和表情代码的网格转换。为优化双映射模块，我们提出了一种文本引导优化方法，利用基于CLIP的目标函数和表情文本提示作为目标，同时整合子空间投影机制，将文本嵌入投影到表情子空间，从而实现对细粒度表情的更精确控制。广泛的实验和对比分析证明了所提方法的有效性和优越性。
英文摘要：
Facial expression editing methods can be mainly categorized into two types based on their architectures: 2D-based and 3D-based methods. The former lacks 3D face modeling capabilities, making it difficult to edit 3D factors effectively. The latter has demonstrated superior performance in generating high-quality and view-consistent renderings using single-view 2D face images. Although these methods have successfully used animatable models to control facial expressions, they still have limitations in achieving precise control over fine-grained expressions. To address this issue, in this paper, we propose a novel approach by simultaneously refining both the latent code of a pretrained 3D-Aware GAN model for texture editing and the expression code of the driven 3DMM model for mesh editing. Specifically, we introduce a Dual Mappers module, comprising Texture Mapper and Emotion Mapper, to learn the transformations of the given latent code for textures and the expression code for meshes, respectively. To optimize the Dual Mappers, we propose a Text-Guided Optimization method, leveraging a CLIP-based objective function with expression text prompts as targets, while integrating a SubSpace Projection mechanism to project the text embedding to the expression subspace such that we can have more precise control over fine-grained expressions. Extensive experiments and comparative analyses demonstrate the effectiveness and superiority of our proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
