PACER：基于自适应长度的块级预验证用于推测解码
PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length
AI总结：
Pacer通过自适应长度的块级预验证技术，提升推测解码效率，实现高达3.09倍的加速。
摘要：
推测解码（SD）是一种强大的技术，用于加速大型语言模型（LLMs）的推理过程，而不会牺牲准确性。通常，SD使用一个小的草稿模型生成固定数量的草稿标记，然后由目标模型并行验证。然而，我们的实验表明，最优的草稿长度在不同的解码步骤中变化显著。这种变化表明，使用固定长度的草稿限制了进一步提高解码速度的潜力。为了解决这一挑战，我们提出了Pacer，一种新颖的方法，通过轻量级、可训练的预验证层动态控制草稿长度。该层在将草稿标记发送到目标模型之前，进行块级预验证，允许草稿模型在块级预验证失败时停止生成标记。我们将在多个SD模型对上实现Pacer，并在各种基准上评估其性能。我们的结果表明，Pacer在自回归解码上实现了高达2.66倍的加速，并且在标准推测解码上表现一致。此外，当与Ouroboros结合时，Pacer实现了高达3.09倍的加速。
英文摘要：
Speculative decoding (SD) is a powerful technique for accelerating the inference process of large language models (LLMs) without sacrificing accuracy. Typically, SD employs a small draft model to generate a fixed number of draft tokens, which are then verified in parallel by the target model. However, our experiments reveal that the optimal draft length varies significantly across different decoding steps. This variation suggests that using a fixed draft length limits the potential for further improvements in decoding speed. To address this challenge, we propose Pacer, a novel approach that dynamically controls draft length using a lightweight, trainable pre-verification layer. This layer pre-verifies draft tokens blockwise before they are sent to the target model, allowing the draft model to stop token generation if the blockwise pre-verification fails. We implement Pacer on multiple SD model pairs and evaluate its performance across various benchmarks. Our results demonstrate that Pacer achieves up to 2.66x Speedup over autoregressive decoding and consistently outperforms standard speculative decoding. Furthermore, when integrated with Ouroboros, Pacer attains up to 3.09x Speedup.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
