用于自回归图像生成的分组投机解码
Grouped Speculative Decoding for Autoregressive Image Generation
AI总结：
本文提出了一种无需训练的加速方法Grouped Speculative Decoding（GSD），通过评估视觉有效token簇而非单个目标token，解决了传统SD在AR图像生成中的假阴性拒绝问题，平均加速3.7倍且保持图像质量。
摘要：
最近，自回归（AR）图像模型展现出了卓越的生成能力，成为扩散模型的一个极具吸引力的替代方案。然而，其序列特性导致推理时间长，限制了实际的可扩展性。在这项工作中，我们引入了Grouped Speculative Decoding（GSD），一种用于AR图像模型的全新、无需训练的加速方法。虽然最近的研究探索了Speculative Decoding（SD）作为加速AR图像生成的手段，但现有方法要么只提供有限的加速，要么需要额外的训练。我们的深入分析揭示了语言和图像token之间的一个根本区别：图像token表现出固有的冗余性和多样性，意味着多个token可以传达有效的语义。然而，传统的SD方法被设计为只接受单个最可能的token，这未能利用这种差异，导致过多的假阴性拒绝。为了解决这个问题，我们提出了一种新的SD策略，该策略评估视觉有效token的簇，而不是依赖单个目标token。此外，我们观察到基于嵌入距离的静态聚类是无效的，这促使我们提出了动态GSD方法。大量实验表明，GSD在保持图像质量的同时，将AR图像模型平均加速了3.7倍——所有这些都无需任何额外的训练。源代码可在https://github.com/junhyukso/GSD获取。
英文摘要：
Recently, autoregressive (AR) image models have demonstrated remarkable generative capabilities, positioning themselves as a compelling alternative to diffusion models. However, their sequential nature leads to long inference times, limiting their practical scalability. In this work, we introduce Grouped Speculative Decoding (GSD), a novel, training-free acceleration method for AR image models. While recent studies have explored Speculative Decoding (SD) as a means to speed up AR image generation, existing approaches either provide only modest acceleration or require additional training. Our in-depth analysis reveals a fundamental difference between language and image tokens: image tokens exhibit inherent redundancy and diversity, meaning multiple tokens can convey valid semantics. However, traditional SD methods are designed to accept only a single most-likely token, which fails to leverage this difference, leading to excessive false-negative rejections. To address this, we propose a new SD strategy that evaluates clusters of visually valid tokens rather than relying on a single target token. Additionally, we observe that static clustering based on embedding distance is ineffective, which motivates our dynamic GSD approach. Extensive experiments show that GSD accelerates AR image models by an average of 3.7x while preserving image quality-all without requiring any additional training. The source code is available at https://github.com/junhyukso/GSD
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
