基于语义引导的两阶段GAN用于面部修复的混合感知编码
Semantic-Guided Two-Stage GAN for Face Inpainting with Hybrid Perceptual Encoding
AI总结：
本文提出一种基于语义引导的两阶段GAN模型，通过混合感知编码提升面部修复的语义一致性和图像质量。
摘要：
面部图像修复的目标是在保持身份、结构一致性和逼真图像质量的前提下，恢复面部图像中缺失或损坏的区域，这一任务专门为照片修复而设计。尽管深度生成模型近年来取得了许多进展，但现有方法在处理大不规则掩码时面临问题，常常在掩码区域边缘产生模糊的纹理，出现语义不一致或不令人信服的面部结构，这主要是由于直接像素级合成方法和对面部先验知识利用有限所致。在本文中，我们提出了一种新的架构，通过语义引导的分层合成来解决上述挑战。我们的方法首先采用一种基于意义组织和合成信息的方法，然后进行纹理细化。这一过程在进一步创建详细图像之前，提供了清晰的面部结构洞察。在第一阶段，我们结合了专注于局部特征的CNN和全局特征的视觉变换器。这帮助我们创建了清晰且详细的语义布局。在第二阶段，我们使用多模态纹理生成器来细化这些布局，通过从不同尺度拉取信息，确保一切看起来连贯且一致。该架构通过动态注意力机制自然处理任意掩码配置，而无需特定于掩码的训练。在CelebA-HQ和FFHQ两个数据集上的实验表明，我们的模型优于其他最先进的方法，在LPIPS、PSNR和SSIM等指标上表现出改进。它在具有挑战性的大面积修复情况下产生了视觉上引人注目的结果，具有更好的语义保持性。
英文摘要：
Facial Image inpainting aim is to restore the missing or corrupted regions in face images while preserving identity, structural consistency and photorealistic image quality, a task specifically created for photo restoration. Though there are recent lot of advances in deep generative models, existing methods face problems with large irregular masks, often producing blurry textures on the edges of the masked region, semantic inconsistencies, or unconvincing facial structures due to direct pixel level synthesis approach and limited exploitation of facial priors. In this paper we propose a novel architecture, which address these above challenges through semantic-guided hierarchical synthesis. Our approach starts with a method that organizes and synthesizes information based on meaning, followed by refining the texture. This process gives clear insights into the facial structure before we move on to creating detailed images. In the first stage, we blend two techniques: one that focuses on local features with CNNs and global features with Vision Transformers. This helped us create clear and detailed semantic layouts. In the second stage, we use a Multi-Modal Texture Generator to refine these layouts by pulling in information from different scales, ensuring everything looks cohesive and consistent. The architecture naturally handles arbitrary mask configurations through dynamic attention without maskspecific training. Experiment on two datasets CelebA-HQ and FFHQ shows that our model outperforms other state-of-the-art methods, showing improvements in metrics like LPIPS, PSNR, and SSIM. It produces visually striking results with better semantic preservation, in challenging large-area inpainting situations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
