ClipFace：带纹理三维可变形模型的文本引导编辑
ClipFace: Text-guided Editing of Textured 3D Morphable Models
AI总结：
本文提出 ClipFace，通过可微渲染、对抗训练与 CLIP 损失自监督联合预测三维可变形人脸模型的纹理和表情潜码，实现文本引导的高质量三维人脸编辑及动态纹理生成。
摘要：
我们提出 ClipFace，这是一种新颖的自监督方法，用于对带纹理的人脸三维可变形模型进行文本引导编辑。具体而言，我们采用易于使用的语言提示，以实现对三维人脸表情及外观的控制。我们利用三维可变形模型的几何表达能力——这类模型本身固有的可控性和纹理表达能力有限——并开发了一个自监督生成模型，以在三维空间中联合合成富有表情、带纹理且可关节运动的人脸。我们通过对抗式自监督训练实现高质量的三维人脸纹理生成，该训练由针对真实 RGB 图像集合的可微渲染引导。语言提示提供了可控的编辑与操纵能力，用于调整三维可变形模型的纹理和表情。为此，我们提出了一个神经网络，可同时预测该可变形模型的纹理潜码和表情潜码。我们的模型以自监督方式训练，利用了可微渲染以及基于预训练 CLIP 模型的损失函数。训练完成后，我们的模型可在单次前向传播中联合预测 UV 空间中的人脸纹理以及表情参数，从而同时捕捉面部表情中的几何变化和纹理变化。我们进一步展示了本方法适用于为给定动画序列生成随时间变化的纹理。
英文摘要：
We propose ClipFace, a novel self-supervised approach for text-guided editing of textured 3D morphable model of faces. Specifically, we employ user-friendly language prompts to enable control of the expressions as well as appearance of 3D faces. We leverage the geometric expressiveness of 3D morphable models, which inherently possess limited controllability and texture expressivity, and develop a self-supervised generative model to jointly synthesize expressive, textured, and articulated faces in 3D. We enable high-quality texture generation for 3D faces by adversarial self-supervised training, guided by differentiable rendering against collections of real RGB images. Controllable editing and manipulation are given by language prompts to adapt texture and expression of the 3D morphable model. To this end, we propose a neural network that predicts both texture and expression latent codes of the morphable model. Our model is trained in a self-supervised fashion by exploiting differentiable rendering and losses based on a pre-trained CLIP model. Once trained, our model jointly predicts face textures in UV-space, along with expression parameters to capture both geometry and texture changes in facial expressions in a single forward pass. We further show the applicability of our method to generate temporally changing textures for a given animation sequence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
