FitDiff：使用扩散模型进行鲁棒的单目3D面部形状与反射率估计
FitDiff: Robust monocular 3D facial shape and reflectance estimation using Diffusion Models
AI总结：
提出首个基于人脸识别嵌入的条件3D潜在扩散模型FitDiff，从单张无约束2D人脸图像准确生成可重新打光的高保真3D面部头像，实现最先进性能。
摘要：
3D人脸重建的显著进展已实现高细节和逼真的人脸表示。近期，扩散模型通过超越GAN的性能，彻底改变了生成方法的能力。在本研究中，我们提出了FitDiff，一种基于扩散的3D面部头像生成模型。利用扩散原理，我们的模型利用从“自然”2D面部图像中提取的身份嵌入，准确地生成可重新打光的面部头像。引入的多模态扩散模型是首个同时输出面部反射率贴图（漫反射和镜面反射反照率及法线）与形状的模型，展现出卓越的泛化能力。它仅在公共面部数据集的标注子集上进行训练，并与3D重建配对。我们通过使用感知损失和人脸识别损失指导反向扩散过程，重新审视了典型的3D面部拟合方法。作为首个以人脸识别嵌入为条件的3D LDM，FitDiff仅从无约束的人脸图像开始，即可重建可重新打光的人类头像，并可直接用于常见渲染引擎，实现了最先进的性能。
英文摘要：
The remarkable progress in 3D face reconstruction has resulted in high-detail and photorealistic facial representations. Recently, Diffusion Models have revolutionized the capabilities of generative methods by surpassing the performance of GANs. In this work, we present FitDiff, a diffusion-based 3D facial avatar generative model. Leveraging diffusion principles, our model accurately generates relightable facial avatars, utilizing an identity embedding extracted from an "in-the-wild" 2D facial image. The introduced multi-modal diffusion model is the first to concurrently output facial reflectance maps (diffuse and specular albedo and normals) and shapes, showcasing great generalization capabilities. It is solely trained on an annotated subset of a public facial dataset, paired with 3D reconstructions. We revisit the typical 3D facial fitting approach by guiding a reverse diffusion process using perceptual and face recognition losses. Being the first 3D LDM conditioned on face recognition embeddings, FitDiff reconstructs relightable human avatars, that can be used as-is in common rendering engines, starting only from an unconstrained facial image, and achieving state-of-the-art performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
