arXiv 2403.176642024-03-26cs.CV

DiffFAE：基于空间敏感定制与语义保留的高保真一次性面部外观编辑

DiffFAE: Advancing High-fidelity One-shot Facial Appearance Editing with Space-sensitive Customization and Semantic Preservation

Tencent Youtu Laboratory（腾讯优图实验室）；Nanjing University（南京大学）；Zhejiang University（浙江大学）；VIVO（维沃（vivo））；Fudan University（复旦大学）

Qilin Wang, Jiangning Zhang, Chengming Xu, Weijian Cao, Ying Tai, Yue Han, Yanhao Ge, Hong Gu, Chengjie Wang, Yanwei Fu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出DiffFAE，一个单阶段高效扩散框架，通过空间敏感物理定制和区域响应语义组合模块及一致性正则化，解决面部外观编辑中的保真度低、属性保留差和推理效率低问题，实现最先进性能。

摘要：

面部外观编辑（FAE）旨在修改人脸图像的姿态、表情和光照等物理属性，同时保留身份和背景等属性，在摄影中具有重要意义。尽管该领域取得了巨大进展，但当前研究普遍面临三个挑战：生成保真度低、属性保留差和推理效率低。为克服上述挑战，本文提出了DiffFAE，这是一个专为高保真FAE定制的单阶段高效扩散框架。为实现高保真查询属性转移，我们采用空间敏感物理定制（SPC），通过利用从3D形变模型（3DMM）导出的渲染纹理来确保保真度和泛化能力。为了保留源属性，我们引入了区域响应语义组合（RSC）。该模块被引导学习解耦的源相关特征，从而更好地保留身份并减轻头发、衣服和背景等非面部属性产生的伪影。我们进一步为流程引入了一致性正则化，利用扩散模型注意力矩阵中的先验知识来增强编辑可控性。大量实验证明了DiffFAE优于现有方法，在面部外观编辑中达到了最先进的性能。

英文摘要：

Facial Appearance Editing (FAE) aims to modify physical attributes, such as pose, expression and lighting, of human facial images while preserving attributes like identity and background, showing great importance in photograph. In spite of the great progress in this area, current researches generally meet three challenges: low generation fidelity, poor attribute preservation, and inefficient inference. To overcome above challenges, this paper presents DiffFAE, a one-stage and highly-efficient diffusion-based framework tailored for high-fidelity FAE. For high-fidelity query attributes transfer, we adopt Space-sensitive Physical Customization (SPC), which ensures the fidelity and generalization ability by utilizing rendering texture derived from 3D Morphable Model (3DMM). In order to preserve source attributes, we introduce the Region-responsive Semantic Composition (RSC). This module is guided to learn decoupled source-regarding features, thereby better preserving the identity and alleviating artifacts from non-facial attributes such as hair, clothes, and background. We further introduce a consistency regularization for our pipeline to enhance editing controllability by leveraging prior knowledge in the attention matrices of diffusion model. Extensive experiments demonstrate the superiority of DiffFAE over existing methods, achieving state-of-the-art performance in facial appearance editing.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。