加速自回归文本到图像生成的投机雅可比去噪解码
Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generation
AI总结：
针对自回归文本到图像模型逐令牌解码导致的慢推理问题，提出投机雅可比去噪解码（SJD2），将去噪过程融入雅可比迭代实现并行令牌生成，通过下一干净令牌预测范式稳定轨迹，在减少前向传播次数的同时保持图像质量。
摘要：
作为视觉内容生成的一种新范式，自回归文本到图像模型因其逐令牌顺序解码过程而遭受推理速度慢的问题，生成单张图像往往需要数千次模型前向传播。为解决这一低效问题，我们提出了投机雅可比去噪解码（SJD2），一种将去噪过程融入雅可比迭代以在自回归模型中实现并行令牌生成的框架。我们的方法引入了一种“下一干净令牌预测”范式，使预训练的自回归模型能够接受噪声扰动的令牌嵌入，并通过低成本微调预测下一干净令牌。这种去噪范式引导模型走向更稳定的雅可比轨迹。在推理过程中，我们的方法用高斯噪声初始化令牌序列，并在嵌入空间中进行迭代的下一干净令牌预测。我们采用概率准则并行验证和接受多个令牌，并通过去噪轨迹为下一次迭代细化未被接受的令牌。实验表明，我们的方法能够通过减少模型前向传播次数来加速生成，同时保持生成图像的视觉质量。
英文摘要：
As a new paradigm of visual content generation, autoregressive text-to-image models suffer from slow inference due to their sequential token-by-token decoding process, often requiring thousands of model forward passes to generate a single image. To address this inefficiency, we propose Speculative Jacobi-Denoising Decoding (SJD2), a framework that incorporates the denoising process into Jacobi iterations to enable parallel token generation in autoregressive models. Our method introduces a next-clean-token prediction paradigm that enables the pre-trained autoregressive models to accept noise-perturbed token embeddings and predict the next clean tokens through low-cost fine-tuning. This denoising paradigm guides the model towards more stable Jacobi trajectories. During inference, our method initializes token sequences with Gaussian noise and performs iterative next-clean-token-prediction in the embedding space. We employ a probabilistic criterion to verify and accept multiple tokens in parallel, and refine the unaccepted tokens for the next iteration with the denoising trajectory. Experiments show that our method can accelerate generation by reducing model forward passes while maintaining the visual quality of generated images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
