DREAM-S: 基于可搜索草稿与目标感知精炼的推测解码用于多模态生成
DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation
AI总结：
提出DREAM-S框架，通过神经架构搜索和目标感知超网训练自动优化草稿模型架构与交互策略，结合注意力熵引导的自适应中间特征蒸馏，实现视觉语言模型的高效推测解码，加速比达3.85倍。
摘要：
推测解码（SD）已被证明是加速大型语言模型（LLM）自回归生成的有效技术，然而其在视觉语言模型（VLM）中的应用仍相对未被探索。我们提出 extit{DREAM-S}，一个专门为VLM中快速高效解码设计的新型SD框架。DREAM-S利用神经架构搜索（NAS）框架与目标感知超网训练，自动识别草稿模型与目标模型之间的最优交互策略，以及最适合底层硬件实现平台的草稿模型架构。此外，DREAM-S还结合了由注意力熵引导的自适应中间特征蒸馏，以实现高效的草稿训练。在一系列成熟的VLM上的实验表明，与标准解码方法相比，DREAM-S实现了高达$3.85 imes$的加速，并显著优于现有的SD基线。代码已公开：https://github.com/SAI-Lab-NYU/DREAM-S。
英文摘要：
Speculative decoding (SD) has proven to be an effective technique for accelerating autoregressive generation in large language models (LLMs) however, its application to vision-language models (VLMs) remains relatively unexplored. We propose~\textit{DREAM-S}, a novel SD framework designed specifically for fast and efficient decoding in VLMs. DREAM-S leverages a neural architecture search (NAS) framework with target-aware supernet training to automatically identify both the optimal interaction strategy between the draft and target models, and the most suitable draft model architecture for the underlying hardware implementation platform. DREAM-S additionally incorporates adaptive intermediate feature distillation, guided by attention entropy, to enable efficient draft training. Experiments on a range of well-established VLMs show that DREAM-S achieves up to a $3.85\times$ speedup compared to standard decoding approaches and significantly outperforms existing SD baselines. The code is publicly available at: https://github.com/SAI-Lab-NYU/DREAM-S .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
