arXiv 2502.024652025-11-26cs.CV

面向面部编辑的图像合成一致性与可控性研究

Towards Consistent and Controllable Image Synthesis for Face Editing

Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu（机器视觉与信号分析中心，信息科技与电气工程学院，奥卢大学）；Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Sciences and Medical Engineering, Southeast University（教育部儿童发展与学习科学重点实验室，生物科学与医学工程学院，东南大学）

Mengting Wei, Tuomas Varanka, Yante Li, Xingxun Jiang, Huai-Qian Khor, Guoying Zhao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

RigFace通过结合SD模型和3D面部模型，实现面部图像的光照、表情和姿态可控，提升身份保持和图像真实度。

摘要：

面部编辑方法对于虚拟化身、数字人类合成和身份保持等任务至关重要，传统上基于生成对抗网络（GAN）技术，但近期研究转向扩散模型，因其在图像重建中表现出色。然而，扩散模型在控制特定属性和保持其他未变属性的一致性，尤其是身份特征方面仍面临挑战。为解决这些问题并使面部图像编辑更加方便，我们提出了一种新的方法，利用稳定扩散（SD）模型和粗略的3D面部模型来控制肖像照片的光照、面部表情和头部姿态。我们发现，这一任务本质上涉及目标背景、身份和面部属性的结合，旨在进行编辑。我们致力于充分解耦这些因素的控制，以实现面部编辑的一致性。具体而言，我们的方法称为RigFace，包含：1）一个空间属性编码器，提供精确且解耦的背景、姿态、表情和光照条件；2）一个高一致性面部融合方法，将身份特征从身份编码器转移到预训练SD模型的去噪UNet；3）一个属性控制器，将这些条件注入到去噪UNet中。我们的模型在身份保持和照片真实度方面与现有面部编辑模型相比具有可比甚至更优的性能。

英文摘要：

Face editing methods, essential for tasks like virtual avatars, digital human synthesis and identity preservation, have traditionally been built upon GAN-based techniques, while recent focus has shifted to diffusion-based models due to their success in image reconstruction. However, diffusion models still face challenges in controlling specific attributes and preserving the consistency of other unchanged attributes especially the identity characteristics. To address these issues and facilitate more convenient editing of face images, we propose a novel approach that leverages the power of Stable-Diffusion (SD) models and crude 3D face models to control the lighting, facial expression and head pose of a portrait photo. We observe that this task essentially involves the combinations of target background, identity and face attributes aimed to edit. We strive to sufficiently disentangle the control of these factors to enable consistency of face editing. Specifically, our method, coined as RigFace, contains: 1) A Spatial Attribute Encoder that provides presise and decoupled conditions of background, pose, expression and lighting; 2) A high-consistency FaceFusion method that transfers identity features from the Identity Encoder to the denoising UNet of a pre-trained SD model; 3) An Attribute Rigger that injects those conditions into the denoising UNet. Our model achieves comparable or even superior performance in both identity preservation and photorealism compared to existing face editing models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。