E2F-Net：通过StyleGAN潜空间实现眼部到面部修复
E2F-Net: Eyes-to-Face Inpainting via StyleGAN Latent Space
AI总结：
本文提出E2F-Net，利用双编码器提取眼周特征并映射至StyleGAN潜空间，通过GAN反演优化实现高质量面部修复，减少训练成本，超越现有方法。
摘要：
面部修复是一种恢复面部图像中缺失或受损区域的技术，对于遮挡场景下的人脸识别以及低质量图像分析等应用至关重要。该过程不仅需要生成逼真的视觉效果，还需保留个体身份特征。本文旨在通过提出一种基于生成对抗网络（GAN）的新模型——眼部到面部网络（E2F-Net），实现仅根据眼周区域修复面部。所提方法使用两个专用编码器从眼周区域提取身份特征和非身份特征。随后，将提取的特征映射到预训练StyleGAN生成器的潜空间中，以利用其最先进的性能以及丰富、多样且富有表现力的潜空间，而无需额外训练。我们进一步改进StyleGAN输出，通过一种新的GAN反演优化技术在潜空间中寻找最优编码。作为次要优势，E2F-Net仅需最少的训练过程，从而降低了计算复杂度。通过大量实验，我们证明了该方法能够高质量地重建整个面部，尽管训练和监督工作显著减少，但仍超越了现有技术。我们基于知名的公共人脸数据集生成了七个眼部到面部数据集，用于训练和验证所提出的方法。代码和数据集均已公开。
英文摘要：
Face inpainting, the technique of restoring missing or damaged regions in facial images, is pivotal for applications like face recognition in occluded scenarios and image analysis with poor-quality captures. This process not only needs to produce realistic visuals but also preserve individual identity characteristics. The aim of this paper is to inpaint a face given periocular region (eyes-to-face) through a proposed new Generative Adversarial Network (GAN)-based model called Eyes-to-Face Network (E2F-Net). The proposed approach extracts identity and non-identity features from the periocular region using two dedicated encoders have been used. The extracted features are then mapped to the latent space of a pre-trained StyleGAN generator to benefit from its state-of-the-art performance and its rich, diverse and expressive latent space without any additional training. We further improve the StyleGAN output to find the optimal code in the latent space using a new optimization for GAN inversion technique. Our E2F-Net requires a minimum training process reducing the computational complexity as a secondary benefit. Through extensive experiments, we show that our method successfully reconstructs the whole face with high quality, surpassing current techniques, despite significantly less training and supervision efforts. We have generated seven eyes-to-face datasets based on well-known public face datasets for training and verifying our proposed methods. The code and datasets are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
