基于GAN反演的多样化图像修复与编辑
Diverse Inpainting and Editing with GAN Inversion
AI总结：
本文提出一种基于GAN反演的编码器与混合网络方法，通过结合擦除图像特征与随机样本特征实现多样化图像修复与编辑，并利用高比率特征解决颜色不一致问题。
摘要：
最近的反演方法表明，真实图像可以被反演到StyleGAN的潜在空间中，并且得益于训练良好的GAN模型中语义丰富的特征表示，可以在这些图像上实现大量编辑。然而，大量研究也表明，由于高保真重建与可编辑性之间的权衡，图像反演具有挑战性。在本文中，我们处理一项更加困难的任务，即将擦除图像反演到GAN的潜在空间以进行真实的修复和编辑。此外，通过用不同的潜在样本增强反演的潜在编码，我们实现了多样化的修复。具体来说，我们提出学习一个编码器和混合网络，将来自擦除图像的编码特征与来自随机样本的StyleGAN映射特征相结合。为了鼓励混合网络利用两个输入，我们通过一种新颖的设置利用生成的数据来训练这些网络。我们还利用更高比率的特征来防止修复部分和未擦除部分之间的颜色不一致。我们进行了大量实验，并将我们的方法与最先进的反演和修复方法进行了比较。定性指标和视觉比较显示出显著的改进。
英文摘要：
Recent inversion methods have shown that real images can be inverted into StyleGAN's latent space and numerous edits can be achieved on those images thanks to the semantically rich feature representations of well-trained GAN models. However, extensive research has also shown that image inversion is challenging due to the trade-off between high-fidelity reconstruction and editability. In this paper, we tackle an even more difficult task, inverting erased images into GAN's latent space for realistic inpaintings and editings. Furthermore, by augmenting inverted latent codes with different latent samples, we achieve diverse inpaintings. Specifically, we propose to learn an encoder and mixing network to combine encoded features from erased images with StyleGAN's mapped features from random samples. To encourage the mixing network to utilize both inputs, we train the networks with generated data via a novel set-up. We also utilize higher-rate features to prevent color inconsistencies between the inpainted and unerased parts. We run extensive experiments and compare our method with state-of-the-art inversion and inpainting methods. Qualitative metrics and visual comparisons show significant improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
