arXiv 2507.130322025-07-17cs.CV

复活掩码自回归建模以实现高效且可扩展的图像生成

Resurrect Mask AutoRegressive Modeling for Efficient and Scalable Image Generation

Nanjing University（南京大学）；Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Innovation Institute（上海创新研究院）；The Chinese University of Hong Kong（香港中文大学）；Shanghai Jiao Tong University（上海交通大学）；University of Minnesota Twin Cities（明尼苏达大学 Twin Cities 分校）

Yi Xin, Le Zhuo, Qi Qin, Siqi Luo, Yuewen Cao, Bin Fu, Yangfan He, Hongsheng Li, Guangtao Zhai, Xiaohong Liu, Peng Gao

备注：: 24 pages, 10 figures, 10 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出MaskGIL模型，通过改进双向LLaMA架构和2D RoPE，以8步推理在ImageNet上达到与AR模型相当的FID 3.71，并支持文本驱动生成和语音到图像转换。

摘要：

自回归（AR）模型在图像生成方面取得了显著进展，其中掩码自回归（MAR）模型因其高效的并行解码而受到关注。然而，与标准AR模型相比，MAR模型传统上表现不佳。本研究改进了MAR架构以提高图像生成质量。我们首先评估各种图像分词器，以确定最有效的一种。随后，我们引入了一种改进的双向LLaMA架构，用双向注意力替换因果注意力，并融入2D RoPE，共同构成了我们的先进模型MaskGIL。MaskGIL从111M扩展到1.4B参数，在ImageNet 256x256基准测试中取得了3.71的FID分数，与最先进的AR模型相当，同时仅需8个推理步骤，而AR模型需要256步。此外，我们开发了一个具有775M参数的文本驱动MaskGIL模型，用于在各种分辨率下根据文本生成图像。除图像生成外，MaskGIL还扩展到加速基于AR的生成，并实现实时语音到图像转换。我们的代码和模型可在https://github.com/synbol/MaskGIL获取。

英文摘要：

AutoRegressive (AR) models have made notable progress in image generation, with Masked AutoRegressive (MAR) models gaining attention for their efficient parallel decoding. However, MAR models have traditionally underperformed when compared to standard AR models. This study refines the MAR architecture to improve image generation quality. We begin by evaluating various image tokenizers to identify the most effective one. Subsequently, we introduce an improved Bidirectional LLaMA architecture by replacing causal attention with bidirectional attention and incorporating 2D RoPE, which together form our advanced model, MaskGIL. Scaled from 111M to 1.4B parameters, MaskGIL achieves a FID score of 3.71, matching state-of-the-art AR models in the ImageNet 256x256 benchmark, while requiring only 8 inference steps compared to the 256 steps of AR models. Furthermore, we develop a text-driven MaskGIL model with 775M parameters for generating images from text at various resolutions. Beyond image generation, MaskGIL extends to accelerate AR-based generation and enable real-time speech-to-image conversion. Our codes and models are available at https://github.com/synbol/MaskGIL.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。