For autoregressive (AR) modeling of high-resolution images, vector quantization (VQ) represents an image as a sequence of discrete codes. A short sequence length is important for an AR model to reduce its computational costs to consider long-range interactions of codes. However, we postulate that previous VQ cannot shorten the code sequence and generate high-fidelity images together in terms of the rate-distortion trade-off. In this study, we propose the two-stage framework, which consists of Residual-Quantized VAE (RQ-VAE) and RQ-Transformer, to effectively generate high-resolution images. Given a fixed codebook size, RQ-VAE can precisely approximate a feature map of an image and represent the image as a stacked map of discrete codes. Then, RQ-Transformer learns to predict the quantized feature vector at the next position by predicting the next stack of codes. Thanks to the precise approximation of RQ-VAE, we can represent a 256$\times$256 image as 8$\times$8 resolution of the feature map, and RQ-Transformer can efficiently reduce the computational costs. Consequently, our framework outperforms the existing AR models on various benchmarks of unconditional and conditional image generation. Our approach also has a significantly faster sampling speed than previous AR models to generate high-quality images.
Not All Image Regions Matter: Masked Vector Quantization for Autoregressive Image Generation
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）
Autoregressive Video Generation without Vector Quantization
Beijing University of Posts and Telecommunications（北京邮电大学）；Key Laboratory of Intelligent Information Processing, ICT, CAS（中国科学院计算技术研究所智能信息处理重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Dalian University of Technology（大连理工大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）
Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantization
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学学院）；University of Technology Sydney（悉尼科技大学）；Meituan（美团）；Georgia Institute of Technology（佐治亚理工学院）
FPQVAR: Floating Point Quantization for Visual Autoregressive Model with FPGA Hardware Co-design
Institute for Artificial Intelligence（人工智能研究院）；School of Integrated Circuits（集成电路学院）；Beijing Advanced Innovation Center for Integrated Circuits（集成电路先进创新中心）；School of Software and Microelectronics（软件与微电子学院）
Self-control: A Better Conditional Mechanism for Masked Autoregressive Model
Nankai University（南开大学）；School of Statistics and Data Science（统计与数据科学学院）；Key Laboratory of Pure Mathematics and Combinatorics, Ministry of Education（教育部纯数学与组合数学重点实验室）
Post-Training Quantization via Residual Truncation and Zero Suppression for Diffusion Models
Department of Artificial Intelligence（人工智能系）；Kyung Hee University（庆尚大学）；Department of Electrical Engineering（电气工程系）；Department of Computer Science（计算机科学系）
Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization
University of California, Berkeley（加州大学伯克利分校）；NVIDIA；Massachusetts Institute of Technology（麻省理工学院）；Amazon（亚马逊）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
MoSa: Motion Generation with Scalable Autoregressive Modeling
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School（人工智能通用基础理论国家重点实验室，北京大学深圳研究生院）；Chongqing University of Technology（重庆理工大学）；Tencent Technology Co., Ltd.（腾讯科技有限公司）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，中国科学院自动化研究所）
Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Shanghai Innovation Institute（上海创新研究院）；Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身人工智能研究院）；Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）
MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；ETH Zurich（苏黎世联邦理工学院）；Institute of Artificial Intelligence, Beihang University（北航人工智能研究院）；Zhongguancun Laboratory（中关村实验室）；Department of Electrical Engineering, The City College（城市学院电子工程系）；Department of Computer Science, the Graduate Center, the City University of New York（纽约城市大学计算机科学系）；School of Computer Science and Control Engineering, Shenzhen University of Advanced Technology（深圳先进技术大学计算机科学与控制工程学院）
Shanghai Innovation Institute（上海创新研究院）；Westlake University（西湖大学）；Zhejiang University（浙江大学）；Fudan University（复旦大学）；JD.COM（京东公司）；University of Chinese Academy of Sciences（中国科学院大学）
PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
MAG: Multi-Modal Aligned Autoregressive Co-Speech Gesture Generation without Vector Quantization
Communication University of China（中国传媒大学）；China Mobile Research Institute（中国移动研究院）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；The Chinese University of Hong Kong（香港中文大学）
