面向GAN逆映射的空间-上下文差异信息补偿
Spatial-Contextual Discrepancy Information Compensation for GAN Inversion
AI总结：
针对GAN逆映射中失真与编辑性难以平衡的问题，提出含差异信息预测与补偿网络的SDIC方法，通过空间-上下文差异补偿提升重建编辑质量，实现快速推理下的性能权衡。
摘要：
现有大多数GAN逆映射方法要么重建精度高但编辑性不足，要么编辑性强却以保真度为代价，因此如何平衡失真与编辑性的权衡是GAN逆映射领域的重大挑战。针对这一问题，本文提出一种基于空间-上下文差异信息补偿的新型GAN逆映射方法SDIC，该方法由差异信息预测网络DIPN和差异信息补偿网络DICN组成，遵循“补偿-编辑”范式，成功弥补了原始图像与重建/编辑图像之间的图像细节差距。一方面，DIPN对原始图像和初始重建图像的多层空间-上下文信息进行编码，再通过两个沙漏模块预测空间-上下文引导的差异图，从而学习到能建模上下文关系、捕捉细粒度图像细节的可靠差异图。另一方面，DICN通过不同变换将预测的差异信息融入潜在编码和GAN生成器，生成高质量的重建/编辑图像，有效补偿了GAN逆映射过程中的图像细节损失。定量与定性实验均表明，本文方法在图像逆映射和编辑任务上，均能以较快的推理速度实现优异的失真-编辑性权衡效果。
英文摘要：
Most existing GAN inversion methods either achieve accurate reconstruction but lack editability or offer strong editability at the cost of fidelity. Hence, how to balance the distortioneditability trade-off is a significant challenge for GAN inversion. To address this challenge, we introduce a novel spatial-contextual discrepancy information compensationbased GAN-inversion method (SDIC), which consists of a discrepancy information prediction network (DIPN) and a discrepancy information compensation network (DICN). SDIC follows a "compensate-and-edit" paradigm and successfully bridges the gap in image details between the original image and the reconstructed/edited image. On the one hand, DIPN encodes the multi-level spatial-contextual information of the original and initial reconstructed images and then predicts a spatial-contextual guided discrepancy map with two hourglass modules. In this way, a reliable discrepancy map that models the contextual relationship and captures finegrained image details is learned. On the other hand, DICN incorporates the predicted discrepancy information into both the latent code and the GAN generator with different transformations, generating high-quality reconstructed/edited images. This effectively compensates for the loss of image details during GAN inversion. Both quantitative and qualitative experiments demonstrate that our proposed method achieves the excellent distortion-editability trade-off at a fast inference speed for both image inversion and editing tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
