连续推测解码用于自回归图像生成
Continuous Speculative Decoding for Autoregressive Image Generation
AI总结：
提出连续推测解码方法，通过近似准则、去噪轨迹对齐和接受-拒绝采样，加速连续视觉自回归模型，实现2倍以上加速且保持生成质量。
摘要：
连续视觉自回归模型在图像生成中展现出有前景的性能，但其固有的顺序性质导致推理速度较慢。推测解码作为大型语言模型的一种成功加速技术，已有效加速了离散视觉自回归模型。然而，由于连续分布缺乏类似理论，无法用于加速连续自回归模型。为填补这一空白，本工作提出了连续推测解码，并解决了以下挑战：1) 目标模型和草稿模型输出的分布不一致导致接受率低；2) 复杂积分导致修改后的分布没有解析表达式。针对挑战1)，我们通过近似准则、基于重参数化邻近性的新颖去噪轨迹对齐策略以及令牌预填充来解决低接受率问题。针对挑战2)，我们引入带有适当上界的接受-拒绝采样算法，从而避免显式计算积分。此外，我们的去噪轨迹对齐也复用于接受-拒绝采样，有效避免了重复的扩散模型推理。在256x256和512x512分辨率的各种模型上的大量实验表明，我们的方法在保持图像生成质量的同时实现了超过2倍的实际加速。代码可在以下网址获取：this https URL
英文摘要：
Continuous visual autoregressive (AR) models have demonstrated promising performance in image generation, but their inherently sequential nature results in slow inference speed. Speculative decoding, a successful acceleration technique for large language models (LLMs), has effectively accelerated discrete visual AR models. However, the absence of an analogous theory for continuous distributions precludes its use in accelerating continuous AR models. To fill this gap, this work presents continuous speculative decoding, and addresses challenges from: 1) low acceptance rate, caused by inconsistent output distribution modeled by target and draft models, and 2) modified distribution without analytic expression, caused by a complex integral. For challenge 1), we address low acceptance rates through an approximated criterion, a novel denoising trajectory alignment strategy based on reparameterization proximity, and token pre-filling. For challenge 2), we introduce acceptance-rejection sampling algorithm with an appropriate upper bound, thereby avoiding explicitly calculating the integral. Furthermore, our denoising trajectory alignment is also reused in acceptance-rejection sampling, effectively avoiding repetitive diffusion model inference. Extensive experiments on various models at 256x256 and 512x512 resolutions demonstrate that our approach achieves over 2x wall-time speedup while preserving the image generation quality. Codes is available at: https://github.com/MarkXCloud/CSpD
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
