arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.11413cs.CVeess.IV
收藏

梯度调整网络的域反演

Gradient Adjusting Networks for Domain Inversion

Blavatnik School of Computer Science, Tel Aviv University(特拉维夫大学布拉瓦特尼克计算机科学学院)

Erez Sheffi, Michael Rotman, Lior Wolf

摘要:

StyleGAN2被证明是一个强大的图像生成引擎,支持语义编辑。然而,为了操纵真实世界的图像,首先需要能够在StyleGAN的潜在空间中检索其对应的潜在表示,该潜在表示被解码为尽可能接近所需图像的图像。对于许多真实世界的图像,潜在的表示并不存在,这就需要调整生成器网络。我们提出了一种逐图像优化方法,该方法调整StyleGAN2生成器,使其实现对生成器权重的本地编辑,从而实现几乎完美的反转,同时仍然允许图像编辑,通过保持输入潜在表示张量和输出图像之间的其余映射相对完整。该方法是基于一组浅层更新网络(aka)的一次性训练。梯度修改模块),修改生成器的层。在训练梯度修改模块之后,通过将这些网络单个应用于原始参数来获得修改的生成器,并且保持生成器的先前编辑能力。我们的实验表明,在这个非常活跃的领域中,与当前最先进的技术相比,性能存在相当大的差距。我们的代码可在\url{https://github.com/sheffier/gani}上获得。

英文摘要:

StyleGAN2 was demonstrated to be a powerful image generation engine that supports semantic editing. However, in order to manipulate a real-world image, one first needs to be able to retrieve its corresponding latent representation in StyleGAN's latent space that is decoded to an image as close as possible to the desired image. For many real-world images, a latent representation does not exist, which necessitates the tuning of the generator network. We present a per-image optimization method that tunes a StyleGAN2 generator such that it achieves a local edit to the generator's weights, resulting in almost perfect inversion, while still allowing image editing, by keeping the rest of the mapping between an input latent representation tensor and an output image relatively intact. The method is based on a one-shot training of a set of shallow update networks (aka. Gradient Modification Modules) that modify the layers of the generator. After training the Gradient Modification Modules, a modified generator is obtained by a single application of these networks to the original parameters, and the previous editing capabilities of the generator are maintained. Our experiments show a sizable gap in performance over the current state of the art in this very active domain. Our code is available at \url{https://github.com/sheffier/gani}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑