arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.04857cs.CV
收藏

自回归图像生成只需少量缓存的token

Autoregressive Image Generation Needs Only a Few Lines of Cached Tokens

Shanghai Jiao Tong University(上海交通大学);Rakuten;Peking University(北京大学)

Ziran Qin, Youru Lv, Mingbao Lin, Zeren Zhang, Chanfan Gan, Tieyuan Chen, Weiyao Lin

AI总结:

LineAR通过行级缓存压缩技术,在自回归图像生成中实现内存节省和吞吐量提升,同时保持生成质量。

摘要:

自回归(AR)视觉生成已作为一种强大的范式,用于图像和多模态合成,得益于其可扩展性和通用性。然而,现有的自回归图像生成面临严重的内存瓶颈,因为解码过程中需要缓存所有先前生成的视觉token,导致存储需求高且吞吐量低。在本文中,我们引入了LineAR,一种新颖的、无需训练的逐步键值(KV)缓存压缩流程,用于自回归图像生成。通过充分利用视觉注意力的内在特性,LineAR在行级别管理缓存,采用二维视角,保留视觉依赖区域,同时逐步驱逐信息量较低的token,这些token对后续行生成无害,由行间注意力引导。LineAR通过仅使用少量缓存行实现了高效的自回归图像生成,实现了内存节省和吞吐量加速,同时保持或甚至提高生成质量。在六个自回归图像生成模型上进行了广泛的实验,包括类别条件和文本到图像生成,验证了其有效性与通用性。LineAR在LlamaGen-XL和Janus-Pro-1B上将ImageNet FID从2.77提升到2.68,将COCO FID从23.85降至22.86,同时仅保留1/6的KV缓存。它还通过仅使用1/8的KV缓存提升了Lumina-mGPT-768上的DPG。此外,LineAR实现了显著的内存和吞吐量增益,包括在LlamaGen-XL上高达67.61%的内存减少和7.57倍的加速,在Janus-Pro-7B上达到39.66%的内存减少和5.62倍的加速。

英文摘要:

Autoregressive (AR) visual generation has emerged as a powerful paradigm for image and multimodal synthesis, owing to its scalability and generality. However, existing AR image generation suffers from severe memory bottlenecks due to the need to cache all previously generated visual tokens during decoding, leading to both high storage requirements and low throughput. In this paper, we introduce \textbf{LineAR}, a novel, training-free progressive key-value (KV) cache compression pipeline for autoregressive image generation. By fully exploiting the intrinsic characteristics of visual attention, LineAR manages the cache at the line level using a 2D view, preserving the visual dependency regions while progressively evicting less-informative tokens that are harmless for subsequent line generation, guided by inter-line attention. LineAR enables efficient autoregressive (AR) image generation by utilizing only a few lines of cache, achieving both memory savings and throughput speedup, while maintaining or even improving generation quality. Extensive experiments across six autoregressive image generation models, including class-conditional and text-to-image generation, validate its effectiveness and generality. LineAR improves ImageNet FID from 2.77 to 2.68 and COCO FID from 23.85 to 22.86 on LlamaGen-XL and Janus-Pro-1B, while retaining only 1/6 KV cache. It also improves DPG on Lumina-mGPT-768 with just 1/8 KV cache. Additionally, LineAR achieves significant memory and throughput gains, including up to 67.61% memory reduction and 7.57x speedup on LlamaGen-XL, and 39.66% memory reduction and 5.62x speedup on Janus-Pro-7B.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑