迈向精确的图像编码：基于动态向量量化的改进自回归图像生成
Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantization
AI总结：
针对现有向量量化自回归模型固定长度编码导致信息密度不匹配的问题，提出DQ-VAE和DQ-Transformer两阶段框架，通过动态可变长度编码和粗到细的自回归生成，显著提升了图像生成的有效性与效率。
摘要：
现有的基于向量量化（VQ）的自回归模型遵循两阶段生成范式，首先学习一个码本将图像编码为离散码，然后基于所学码本完成生成。然而，它们将固定大小的图像区域编码为固定长度的码，忽略了其天然不同的信息密度，这导致重要区域的信息不足而非重要区域存在冗余，最终降低了生成质量和速度。此外，固定长度编码导致了不自然的光栅扫描自回归生成。为解决该问题，我们提出了一种新颖的两阶段框架：（1）动态量化变分自编码器（DQ-VAE），它根据图像区域的信息密度将其编码为可变长度的码，以实现准确且紧凑的码表示。（2）DQ-Transformer，它通过新颖的堆叠 Transformer 架构以及共享内容、非共享位置输入层设计，交替地对每个粒度下码的位置和内容进行建模，从而从粗粒度（使用较少码的平滑区域）到细粒度（使用较多码的细节区域）自回归地生成图像。在各种生成任务上的综合实验验证了我们在有效性和效率上的优越性。代码将发布于 https://github.com/CrossmodalGroup/DynamicVectorQuantization。
英文摘要：
Existing vector quantization (VQ) based autoregressive models follow a two-stage generation paradigm that first learns a codebook to encode images as discrete codes, and then completes generation based on the learned codebook. However, they encode fixed-size image regions into fixed-length codes and ignore their naturally different information densities, which results in insufficiency in important regions and redundancy in unimportant ones, and finally degrades the generation quality and speed. Moreover, the fixed-length coding leads to an unnatural raster-scan autoregressive generation. To address the problem, we propose a novel two-stage framework: (1) Dynamic-Quantization VAE (DQ-VAE) which encodes image regions into variable-length codes based on their information densities for an accurate and compact code representation. (2) DQ-Transformer which thereby generates images autoregressively from coarse-grained (smooth regions with fewer codes) to fine-grained (details regions with more codes) by modeling the position and content of codes in each granularity alternately, through a novel stacked-transformer architecture and shared-content, non-shared position input layers designs. Comprehensive experiments on various generation tasks validate our superiorities in both effectiveness and efficiency. Code will be released at https://github.com/CrossmodalGroup/DynamicVectorQuantization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
