MagicFace: 通过动作单元控制实现高保真的面部表情编辑
MagicFace: High-Fidelity Facial Expression Editing with Action-Unit Control
AI总结：
MagicFace通过动作单元控制实现高保真的面部表情编辑，结合扩散模型和ID编码器，能够精细调整表情并保持身份和背景一致性。
摘要：
我们通过控制同一人面部动作单元(AU)的相对变化来解决面部表情编辑问题。这使我们能够以细粒度、连续和可解释的方式编辑特定人的表情，同时保持其身份、姿态、背景和详细的面部属性。我们模型的核心是基于AU变化的扩散模型和一个用于保持面部细节一致性的ID编码器。具体来说，为了保持输入身份的面部细节，我们利用预训练的Stable-Diffusion模型的力量，并设计了一个ID编码器通过自注意力机制融合外观特征。为了保持背景和姿态的一致性，我们引入了一个高效的属性控制器，通过显式告知模型目标的当前背景和姿态。通过将AU变化注入到去噪UNet中，我们的模型可以使用各种AU组合来动画任意身份，从而在高保真的表情编辑方面优于其他面部表情编辑工作。代码可在https://github.com/weimengting/MagicFace上公开获取。
英文摘要：
We address the problem of facial expression editing by controling the relative variation of facial action-unit (AU) from the same person. This enables us to edit this specific person's expression in a fine-grained, continuous and interpretable manner, while preserving their identity, pose, background and detailed facial attributes. Key to our model, which we dub MagicFace, is a diffusion model conditioned on AU variations and an ID encoder to preserve facial details of high consistency. Specifically, to preserve the facial details with the input identity, we leverage the power of pretrained Stable-Diffusion models and design an ID encoder to merge appearance features through self-attention. To keep background and pose consistency, we introduce an efficient Attribute Controller by explicitly informing the model of current background and pose of the target. By injecting AU variations into a denoising UNet, our model can animate arbitrary identities with various AU combinations, yielding superior results in high-fidelity expression editing compared to other facial expression editing works. Code is publicly available at https://github.com/weimengting/MagicFace.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
