VVS：通过部分验证跳过重加速视觉自回归生成的推测解码
VVS: Accelerating Speculative Decoding for Visual Autoregressive Generation via Partial Verification Skipping
AI总结：
针对视觉自回归生成中推测解码加速受限的问题，本文提出VVS框架，通过部分验证跳过及三个配套模块削减目标模型前向传播次数，在保持生成质量的同时实现2.8倍加速，取得更优的速度-质量权衡。
摘要：
视觉自回归（AR）生成模型在图像生成领域已展现出强大潜力，但其下一token预测范式会带来较高的推理延迟。尽管推测解码（SD）已被证实可有效加速视觉AR模型，但其“单步草稿、单步验证”的范式无法直接减少前向传播次数，限制了加速上限。受视觉token可互换性启发，本文首次探索在视觉AR模型生成的SD流程中跳过验证环节，以直接削减目标模型前向传播次数、降低推理延迟。通过分析草稿阶段的特征，我们发现验证冗余性与陈旧特征可复用性是免验证步骤维持生成质量与加速比的关键因素。基于这两点发现，我们提出新型SD框架VVS，通过部分验证跳过实现视觉AR生成加速，该框架集成三个互补模块：（1）带动态截断的免验证token选择器，（2）token级特征缓存与复用模块，（3）细粒度跳过步骤调度器。最终，VVS在保持有竞争力生成质量的前提下，相比原生AR解码将目标模型前向传播次数降低至原来的1/2.8，相比传统SD框架实现了更优的速度-质量权衡，展现出重构SD范式的巨大潜力。
英文摘要：
Visual autoregressive (AR) generation models have demonstrated strong potential for image generation, yet their next-token-prediction paradigm introduces considerable inference latency. Although speculative decoding (SD) has been proven effective for accelerating visual AR models, its "draft one step, then verify one step" paradigm prevents a direct reduction of the forward passes, thus restricting acceleration potential. Motivated by the visual token interchangeability, we for the first time to explore verification skipping in the SD process of visual AR model generation to explicitly cut the number of target model forward passes, thereby reducing inference latency. Based on an analysis of the drafting stage's characteristics, we observe that verification redundancy and stale feature reusability are key factors to retain generation quality and speedup for verification-free steps. Inspired by these two observations, we propose a novel SD framework VVS to accelerate visual AR generation via partial verification skipping, which integrates three complementary modules: (1) a verification-free token selector with dynamical truncation, (2) token-level feature caching and reuse, and (3) fine-grained skipped step scheduling. Consequently, VVS reduces the number of target model forward passes by a factor of $2.8\times$ relative to vanilla AR decoding while maintaining competitive generation quality, offering a superior speed-quality trade-off over conventional SD frameworks and revealing strong potential to reshape the SD paradigm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
