基于可逆性分解的域外 GAN 反演用于照片级真实感人脸操控
Out-of-domain GAN inversion via Invertibility Decomposition for Photo-Realistic Human Face Manipulation
AI总结：
针对 GAN 人脸反演中 OOD 区域导致掩码不准的问题，提出联合学习可逆性检测与空间对齐的分解框架，提升反演和属性操控保真度。
摘要：
生成对抗网络（GAN）反演的保真度会受到图像中域外（OOD）区域（例如背景、配饰）的阻碍。检测超出预训练模型生成能力的 OOD 区域，并将这些区域与输入图像混合，可以提升保真度。“可逆性掩码”能够识别这些 OOD 区域，而现有方法通过重建误差来预测该掩码。然而，由于域内（ID）区域中重建误差的影响，估计出的掩码通常并不准确。本文提出一种新颖框架，通过设计一个新模块，利用可逆性掩码将输入图像分解为 ID 和 OOD 两个部分，从而提升人脸反演的保真度。与以往工作不同，我们的可逆性检测器与空间对齐模块同时学习。我们迭代地将生成特征与输入几何进行对齐，并降低 ID 区域内的重建误差。因此，OOD 区域更加可区分，并能够被精确预测。随后，我们将来自输入图像的 OOD 区域与 ID 的 GAN 反演结果混合，以提升结果的保真度。对于真实世界的人脸图像反演与操控，我们的方法能够生成照片级真实感的结果。大量实验表明，在 GAN 反演和属性操控质量方面，我们的方法优于现有方法。
英文摘要：
The fidelity of Generative Adversarial Networks (GAN) inversion is impeded by Out-Of-Domain (OOD) areas (e.g., background, accessories) in the image. Detecting the OOD areas beyond the generation ability of the pre-trained model and blending these regions with the input image can enhance fidelity. The "invertibility mask" figures out these OOD areas, and existing methods predict the mask with the reconstruction error. However, the estimated mask is usually inaccurate due to the influence of the reconstruction error in the In-Domain (ID) area. In this paper, we propose a novel framework that enhances the fidelity of human face inversion by designing a new module to decompose the input images to ID and OOD partitions with invertibility masks. Unlike previous works, our invertibility detector is simultaneously learned with a spatial alignment module. We iteratively align the generated features to the input geometry and reduce the reconstruction error in the ID regions. Thus, the OOD areas are more distinguishable and can be precisely predicted. Then, we improve the fidelity of our results by blending the OOD areas from the input image with the ID GAN inversion results. Our method produces photo-realistic results for real-world human face image inversion and manipulation. Extensive experiments demonstrate our method's superiority over existing methods in the quality of GAN inversion and attribute manipulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
