LinkGAN:将GAN潜变量与像素相连接以实现可控图像合成
LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis
AI总结:
LinkGAN提出一种GAN训练正则化器,将潜空间轴与图像像素区域显式连接,实现局部可控生成,并支持多区域联合控制及2D/3D模型,同时兼容GAN反演。
摘要:
本文提出了一种易于使用的GAN训练正则化器,它有助于将潜空间的某些轴与合成图像中的一组像素显式地连接起来。建立这种连接有助于更方便地对GAN生成进行局部控制,用户只需对潜码进行部分重采样,就可以仅在某个空间区域内改变图像内容。实验结果证实了我们称之为LinkGAN的正则化器具有四个吸引人的特性。(1)潜变量-像素连接既适用于固定区域(即对所有实例都相同),也适用于特定的语义类别(即在不同实例间变化),例如天空。(2)两个或多个区域可以独立地连接到不同的潜变量轴上,从而进一步支持联合控制。(3)我们的正则化器能够提升2D和3D感知GAN模型的空间可控性,同时几乎不牺牲合成性能。(4)使用我们的正则化器训练的模型与GAN反演技术兼容,并保持对真实图像的可编辑性。
英文摘要:
This work presents an easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image. Establishing such a connection facilitates a more convenient local control of GAN generation, where users can alter the image content only within a spatial area simply by partially resampling the latent code. Experimental results confirm four appealing properties of our regularizer, which we call LinkGAN. (1) The latent-pixel linkage is applicable to either a fixed region (\textit{i.e.}, same for all instances) or a particular semantic category (i.e., varying across instances), like the sky. (2) Two or multiple regions can be independently linked to different latent axes, which further supports joint control. (3) Our regularizer can improve the spatial controllability of both 2D and 3D-aware GAN models, barely sacrificing the synthesis performance. (4) The models trained with our regularizer are compatible with GAN inversion techniques and maintain editability on real images.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。