SpecBound: 带层间置信度校准的自适应有界自我猜测
SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration
AI总结：
本文提出一种新型自适应自我猜测框架，通过层间温度退火抑制虚假置信，根据token解码难度动态限制猜测长度，提升大语言模型的自回归推理效率。
摘要：
自我猜测解码已逐渐成为加速大语言模型（LLMs）自回归推理的有前景方法。自我草稿方法利用基础LLM本身进行猜测，避免了辅助草稿模型的开销，但存在局限：浅层往往产生过度自信但错误的token预测，且草稿序列中存在困难token会迫使更深层的计算，影响草稿接受和整体加速。为解决这些问题，我们提出了一种新的自草稿框架，通过层间温度退火抑制虚假置信，并根据token解码难度自适应限制猜测长度。通过重新处理草稿token的隐藏状态，在统一的并行通过深层层中，我们的方法保持与原始模型精确的输出等价性，同时最大化计算效率。它不需要修改基础LLM的参数，并在多样化的长形式生成任务和多种模型架构上实现了高达2.33倍的墙时间加速。
英文摘要：
Speculative decoding has emerged as a promising approach to accelerate autoregressive inference in large language models (LLMs). Self-draft methods, which leverage the base LLM itself for speculation, avoid the overhead of auxiliary draft models but face limitations: shallow layers often produce overconfident yet incorrect token predictions, and the presence of difficult tokens in a draft sequence forces redundant computation through deeper layers, undermining both draft acceptance and overall speedup. To address these issues, we propose a novel self-draft framework that suppresses spurious confidence via layer-wise temperature annealing in early-exit decision and adaptively bounds speculation length based on token-wise decoding difficulty. By reprocessing the hidden states of draft tokens in a unified parallel pass through deep layers, our method maintains exact output equivalence with the original model while maximizing computational efficiency. It requires no modifications to the base LLM parameters and achieves up to 2.33x wall-time speedup over standard autoregressive decoding across diverse long-form generation tasks and multiple model architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
