SENSE: 基于软门控评估的语义嵌入导航用于检索式推测解码
SENSE: Semantic Embedding Navigation with Soft-gated Evaluation for Retrieval-based Speculative Decoding
AI总结：
提出SENSE方法，通过语义嵌入导航和软门控评估模块替代表面形式匹配，提升检索式推测解码的鲁棒性和加速效果，在LLaMA和Qwen系列上实现最高4.09平均接受长度和3.26倍加速。
摘要：
推测解码（SD）通过使用轻量级草稿模型提出候选令牌，并由目标模型并行验证，从而加速大型语言模型（LLM）推理，同时不损害生成质量。尽管检索式推测解码（RSD）因其即插即用的多功能性而受到青睐，但其潜力受到刚性词汇依赖的阻碍，使得检索和验证对表面形式变化敏感。为了解决这个问题，我们提出了SENSE（基于软门控评估的语义嵌入导航）。通过将检索锚定在目标模型的隐藏状态上，SENSE建立了稳健的语义对齐，这使得软门控评估模块能够验证语义等价性而非表面形式。为了确保严格的基准测试，我们将现有方法解构为统一框架内的原子原语，促进细粒度的组件级比较。跨多个领域的广泛实验表明，SENSE在LLaMA和Qwen系列上优于多个基线，实现了高达4.09的平均接受长度和3.26倍的加速，同时保持了生成质量。我们的代码将在发表后发布。
英文摘要：
Speculative Decoding (SD) accelerates Large Language Model (LLM) inference by employing a lightweight draft model to propose candidate tokens, which are verified in parallel by the target model, without compromising generation quality. While Retrieval-based Speculative Decoding (RSD) is favored for its plug-and-play versatility, its potential is impeded by rigid lexical dependencies, rendering both retrieval and verification brittle to surface-level variations. To address this, we propose SENSE (Semantic Embedding Navigation with Soft-gated Evaluation). By anchoring retrieval on the hidden states of the target model, SENSE establishes robust semantic alignment, which empowers the Soft-gated Evaluation module to validate semantic equivalence rather than surface forms. To ensure rigorous benchmarking, we deconstruct existing methods into atomic primitives within a unified framework, facilitating granular, component-level comparison. Extensive experiments across diverse domains demonstrate that SENSE outperforms multiple baselines on the LLaMA and Qwen families, attaining up to 4.09 mean acceptance length and 3.26x speedup, while preserving generation quality. Our code will be released upon publication.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
