使用空间填充向量量化对 GAN 潜在空间进行无监督全景解释
Unsupervised Panoptic Interpretation of Latent Spaces in GANs Using Space-Filling Vector Quantization
AI总结：
本文提出空间填充向量量化（SFVQ）方法，以无监督方式建模并解释 GAN 潜在空间，在 StyleGAN2 和 BigGAN 上验证其可识别生成因素、发现可解释变换方向并支持可控数据增强。
摘要：
生成对抗网络（GAN）学习到的潜在空间中的样本可以被映射到真实图像，但这类潜在空间难以解释。一些早期的监督方法旨在创建可解释的潜在空间或发现可解释的方向，这需要利用数据标签或标注的合成样本来进行训练。然而，我们提出使用一种称为空间填充向量量化（SFVQ）的向量量化改进方法，该方法在分段线性曲线上对数据进行量化。SFVQ 能够捕捉潜在空间的底层形态结构，从而使其可解释。我们将该技术应用于在各种数据集上预训练的 StyleGAN2 和 BigGAN 网络的潜在空间建模。实验表明，SFVQ 曲线能够生成潜在空间的通用可解释模型，从而确定潜在空间的哪些部分对应于特定的生成因素。此外，我们证明 SFVQ 曲线的每条线都可能指向一个可解释的方向，用于应用可理解的图像变换。我们还证明，位于 SFVQ 线上的点可用于可控的数据增强。
英文摘要：
Generative adversarial networks (GANs) learn a latent space whose samples can be mapped to real-world images. Such latent spaces are difficult to interpret. Some earlier supervised methods aim to create an interpretable latent space or discover interpretable directions, which requires exploiting data labels or annotated synthesized samples for training. However, we propose using a modification of vector quantization called space-filling vector quantization (SFVQ), which quantizes the data on a piece-wise linear curve. SFVQ can capture the underlying morphological structure of the latent space, making it interpretable. We apply this technique to model the latent space of pre-trained StyleGAN2 and BigGAN networks on various datasets. Our experiments show that the SFVQ curve yields a general interpretable model of the latent space such that it determines which parts of the latent space correspond to specific generative factors. Furthermore, we demonstrate that each line of the SFVQ curve can potentially refer to an interpretable direction for applying intelligible image transformations. We also demonstrate that the points located on an SFVQ line can be used for controllable data augmentation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
