DisControlFace：向扩散自编码器添加解耦控制以实现单样本显式人脸图像编辑
DisControlFace: Adding Disentangled Control to Diffusion Autoencoder for One-shot Explicit Facial Image Editing
AI总结：
本文提出DisControlFace框架，通过冻结Diff-AE预训练权重并结合随机语义掩码策略独立训练Exp-FaceNet，实现了高层语义与3DMM显式参数的解耦控制，在单样本场景下完成了高保真且身份保持更好的显式人脸图像编辑。
摘要：
在本工作中，我们专注于探索生成式人脸图像编辑的显式细粒度控制，同时生成忠实的人脸外观和一致的语义细节。然而这极具挑战性且尚未被广泛探索，尤其是在单样本场景下。我们将关键挑战确定为在生成过程中探索高层语义与显式参数（例如3DMM）之间的解耦条件控制，并相应提出了一种新颖的基于扩散的编辑框架，名为DisControlFace。具体而言，我们利用Diffusion Autoencoder（Diff-AE）作为语义重建主干。为了实现显式人脸编辑，我们构建了一个与Diff-AE兼容的Exp-FaceNet，基于估计的3DMM参数生成空间维度的显式控制条件。与当前从头训练整个条件生成模型的基于扩散的编辑方法不同，我们冻结Diff-AE的预训练权重以维持其语义确定性条件控制能力，并相应提出随机语义掩码（RSM）策略，以有效实现Exp-FaceNet的独立训练。这一设置赋予模型解耦的人脸控制能力，同时减少了编辑中的语义信息偏移。我们的模型可以使用2D自然场景肖像图像进行训练，无需3D或视频数据，并可通过简单的单样本微调对任意新的人脸图像执行鲁棒的编辑。综合实验表明，DisControlFace能够生成逼真的人脸图像，在编辑准确性和身份保持方面优于state-of-the-art方法。项目页面：https://discontrolface.github.io/
英文摘要：
In this work, we focus on exploring explicit fine-grained control of generative facial image editing, all while generating faithful facial appearances and consistent semantic details, which however, is quite challenging and has not been extensively explored, especially under an one-shot scenario. We identify the key challenge as the exploration of disentangled conditional control between high-level semantics and explicit parameters (e.g., 3DMM) in the generation process, and accordingly propose a novel diffusion-based editing framework, named DisControlFace. Specifically, we leverage a Diffusion Autoencoder (Diff-AE) as the semantic reconstruction backbone. To enable explicit face editing, we construct an Exp-FaceNet that is compatible with Diff-AE to generate spatial-wise explicit control conditions based on estimated 3DMM parameters. Different from current diffusion-based editing methods that train the whole conditional generative model from scratch, we freeze the pre-trained weights of the Diff-AE to maintain its semantically deterministic conditioning capability and accordingly propose a random semantic masking (RSM) strategy to effectively achieve an independent training of Exp-FaceNet. This setting endows the model with disentangled face control meanwhile reducing semantic information shift in editing. Our model can be trained using 2D in-the-wild portrait images without requiring 3D or video data and perform robust editing on any new facial image through a simple one-shot fine-tuning. Comprehensive experiments demonstrate that DisControlFace can generate realistic facial images with better editing accuracy and identity preservation over state-of-the-art methods. Project page: https://discontrolface.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
