ASPD：通过探索LLM中的内在并行性解锁自适应串并行解码
ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMs
AI总结：
针对LLM自回归解码的推理延迟问题，提出自适应串并行解码方法ASPD，通过提取模型输出的内在并行结构并结合混合解码引擎，在保持生成质量的同时实现显著推理加速。
摘要：
大型语言模型（LLM）日益增加的规模和复杂性带来了显著的推理延迟挑战，这主要归因于其以下一个token预测的顺序性为特征的自回归解码范式。通过重新审视自回归模型的输出，我们观察到某些片段展现出可并行化的结构，我们将其称为内在并行性。同时解码每个可并行化的分支（即并行解码）可以显著提高LLM的整体推理速度。在本文中，我们提出了一种自适应串并行解码（ASPD），该方法解决了两个核心挑战：可并行化数据的自动构建和高效的并行解码机制。更具体地说，我们引入了一个非侵入式流水线，能够自动从自回归模型的回复中提取并验证可并行化结构。为了实现高效的自适应串并行解码，我们实现了一个混合解码引擎，该引擎能够在串行和并行解码模式之间无缝切换，同时维护可重用的KV缓存，从而最大化计算效率。在通用任务、检索增强生成和数学推理领域的广泛评估表明，ASPD在有效性和效率方面均取得了前所未有的性能。值得注意的是，在Vicuna Bench上，我们的方法实现了高达3.19倍的加速（平均1.85倍），同时与自回归模型相比，回复质量保持在1%的差异以内，在不牺牲生成质量的情况下实现了显著加速。我们的框架为高效的LLM并行推理设定了开创性的基准，为其在AI驱动的客服机器人和答案检索引擎等延迟敏感应用中的部署铺平了道路。
英文摘要：
The increasing scale and complexity of large language models (LLMs) pose significant inference latency challenges, primarily due to their autoregressive decoding paradigm characterized by the sequential nature of next-token prediction. By re-examining the outputs of autoregressive models, we observed that some segments exhibit parallelizable structures, which we term intrinsic parallelism. Decoding each parallelizable branch simultaneously (i.e. parallel decoding) can significantly improve the overall inference speed of LLMs. In this paper, we propose an Adaptive Serial-Parallel Decoding (ASPD), which addresses two core challenges: automated construction of parallelizable data and efficient parallel decoding mechanism. More specifically, we introduce a non-invasive pipeline that automatically extracts and validates parallelizable structures from the responses of autoregressive models. To empower efficient adaptive serial-parallel decoding, we implement a Hybrid Decoding Engine which enables seamless transitions between serial and parallel decoding modes while maintaining a reusable KV cache, maximizing computational efficiency. Extensive evaluations across General Tasks, Retrieval-Augmented Generation, Mathematical Reasoning, demonstrate that ASPD achieves unprecedented performance in both effectiveness and efficiency. Notably, on Vicuna Bench, our method achieves up to 3.19x speedup (1.85x on average) while maintaining response quality within 1% difference compared to autoregressive models, realizing significant acceleration without compromising generation quality. Our framework sets a groundbreaking benchmark for efficient LLM parallel inference, paving the way for its deployment in latency-sensitive applications such as AI-powered customer service bots and answer retrieval engines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
