MoVQ：调制量化矢量以生成高保真图像
MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generation
摘要：
虽然两阶段矢量量化（VQ）生成模型允许合成高保真度和高分辨率图像，但是它们的量化运算符将图像内的相似块编码到相同索引中，从而导致使用现有解码器架构的相似相邻区域的重复伪影。为了解决这个问题，我们建议将空间条件归一化调制的量化矢量，以便插入空间变化的信息嵌入到索引图，鼓励解码器生成更多的照片逼真的图像。此外，我们使用多通道量化来增加离散码的重组能力，而不增加模型和码书的成本。此外，为了在第二阶段生成离散令牌，我们采用了掩码生成图像Transformer（MaskGIT）来学习压缩潜在空间中的底层先验分布，这比传统的自回归模型快得多。两个基准数据集上的实验表明，我们提出的调制VQGAN能够大大提高重建图像的质量，以及提供高保真图像生成。
英文摘要：
Although two-stage Vector Quantized (VQ) generative models allow for synthesizing high-fidelity and high-resolution images, their quantization operator encodes similar patches within an image into the same index, resulting in a repeated artifact for similar adjacent regions using existing decoder architectures. To address this issue, we propose to incorporate the spatially conditional normalization to modulate the quantized vectors so as to insert spatially variant information to the embedded index maps, encouraging the decoder to generate more photorealistic images. Moreover, we use multichannel quantization to increase the recombination capability of the discrete codes without increasing the cost of model and codebook. Additionally, to generate discrete tokens at the second stage, we adopt a Masked Generative Image Transformer (MaskGIT) to learn an underlying prior distribution in the compressed latent space, which is much faster than the conventional autoregressive model. Experiments on two benchmark datasets demonstrate that our proposed modulated VQGAN is able to greatly improve the reconstructed image quality as well as provide high-fidelity image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
