Fast-ARDiff: 一种基于熵信息的连续空间自回归生成加速框架
Fast-ARDiff: An Entropy-informed Acceleration Framework for Continuous Space Autoregressive Generation
AI总结：
Fast-ARDiff通过联合优化AR和扩散组件，结合熵信息引导策略和动态调度器，实现高效连续空间自回归生成加速。
摘要：
自回归（AR）-扩散混合范式结合了AR的结构化建模与扩散的逼真合成，但因顺序AR生成和迭代去噪导致高延迟。在本文中，我们解决这一瓶颈，提出了一种统一的AR-扩散框架Fast-ARDiff，联合优化两个组件，加速AR推测解码的同时也促进更快的扩散解码。具体而言：（1）熵信息引导的推测策略鼓励草稿模型生成与目标模型熵特征一致的高熵表示，缓解因草稿过度自信导致的熵不匹配和高拒绝率。（2）对于扩散解码，而不是将其视为独立模块，我们将其整合到同一个端到端框架中，使用动态调度器优先优化AR，以指导后续步骤的扩散部分。通过结合轨迹和分布匹配的联合蒸馏框架优化扩散部分，确保稳定训练和高质量合成，仅用极少数步骤即可实现。在推理过程中，使用AR模块的浅层特征熵对低熵草稿进行预过滤，避免冗余计算并提高延迟。Fast-ARDiff在多种模型上实现了最先进的加速：在ImageNet 256×256上，TransDiff实现了4.3倍无损加速，NextStep-1在文本条件生成上实现了3倍加速。代码将在https://github.com/aSleepyTree/Fast-ARDiff上提供。
英文摘要：
Autoregressive(AR)-diffusion hybrid paradigms combine AR's structured modeling with diffusion's photorealistic synthesis, yet suffer from high latency due to sequential AR generation and iterative denoising. In this work, we tackle this bottleneck and propose a unified AR-diffusion framework Fast-ARDiff that jointly optimizes both components, accelerating AR speculative decoding while simultaneously facilitating faster diffusion decoding. Specifically: (1) The entropy-informed speculative strategy encourages draft model to produce higher-entropy representations aligned with target model's entropy characteristics, mitigating entropy mismatch and high rejection rates caused by draft overconfidence. (2) For diffusion decoding, rather than treating it as an independent module, we integrate it into the same end-to-end framework using a dynamic scheduler that prioritizes AR optimization to guide the diffusion part in further steps. The diffusion part is optimized through a joint distillation framework combining trajectory and distribution matching, ensuring stable training and high-quality synthesis with extremely few steps. During inference, shallow feature entropy from AR module is used to pre-filter low-entropy drafts, avoiding redundant computation and improving latency. Fast-ARDiff achieves state-of-the-art acceleration across diverse models: on ImageNet 256$\times$256, TransDiff attains 4.3$\times$ lossless speedup, and NextStep-1 achieves 3$\times$ acceleration on text-conditioned generation. Code will be available at https://github.com/aSleepyTree/Fast-ARDiff.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
