基于风格融合的高保真换脸
High-Fidelity Face Swapping with Style Blending
AI总结：
针对传统GAN换脸方法融合不一致、伪影多、训练不稳定等问题，提出基于StyleGAN的端到端高保真换脸框架，通过属性编码、注意力风格融合及多约束优化，在CelebA-HQ数据集上实现了优于现有方法的视觉效果。
摘要：
在深度学习方法推动的人脸合成技术大量涌现的驱动下，换脸技术获得了广泛关注。然而，以往以生成对抗网络（GANs）为主干的换脸方法存在融合不一致、失真、伪影以及训练稳定性等问题。为解决这些局限，我们提出了一种创新的端到端高保真换脸框架。首先，我们引入了基于StyleGAN的人脸属性编码器，用于从人脸中提取关键特征并将其反演为潜在风格编码，封装换脸成功所需的不可或缺的人脸属性。其次，我们提出了基于注意力的风格融合模块，以有效地将源人脸的Face ID迁移到目标人脸。为确保迁移的准确性和质量，我们实施了一系列约束措施，包括对比人脸ID学习、人脸关键点对齐和双重交换一致性。最后，融合后的风格编码通过风格解码器被转换回图像空间，该解码器具有高训练稳定性和生成能力。在CelebA-HQ数据集上开展的大量实验表明，与其他现有最优方法相比，我们的换脸方法生成的图像视觉质量更优，且每个提出的模块均具备有效性。源代码和权重将公开发布。
英文摘要：
Face swapping has gained significant traction, driven by the plethora of human face synthesis facilitated by deep learning methods. However, previous face swapping methods that used generative adversarial networks (GANs) as backbones have faced challenges such as inconsistency in blending, distortions, artifacts, and issues with training stability. To address these limitations, we propose an innovative end-to-end framework for high-fidelity face swapping. First, we introduce a StyleGAN-based facial attributes encoder that extracts essential features from faces and inverts them into a latent style code, encapsulating indispensable facial attributes for successful face swapping. Second, we introduce an attention-based style blending module to effectively transfer Face IDs from source to target. To ensure accurate and quality transferring, a series of constraint measures including contrastive face ID learning, facial landmark alignment, and dual swap consistency is implemented. Finally, the blended style code is translated back to the image space via the style decoder, which is of high training stability and generative capability. Extensive experiments on the CelebA-HQ dataset highlight the superior visual quality of generated images from our face-swapping methodology when compared to other state-of-the-art methods, and the effectiveness of each proposed module. Source code and weights will be publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
