AHASD：异步异构架构用于移动设备上的LLM自适应草案推测解码
AHASD: Asynchronous Heterogeneous Architecture for LLM Adaptive Drafting Speculative Decoding on Mobile Devices
AI总结：
AHASD提出一种异步异构架构，通过任务级DLM-TLM解耦和熵历史感知草案控制，提升移动设备上LLM自适应草案推测解码的吞吐量和能效。
摘要：
推测解码通过使用小型草案语言模型（DLM）生成草案并用大型目标语言模型（TLM）批量验证，从而提高大语言模型（LLM）的推理效率。然而，传统操作级同步执行在移动单NPU-PIM系统上导致空闲开销，异步执行因草案长度波动造成计算浪费。本文提出AHASD，一种任务级异步移动NPU-PIM异构架构用于推测解码。值得注意的是，AHASD通过任务级DLM-TLM解耦实现PIM上的并行草案生成和单NPU上的验证，并且特别集成了熵历史感知草案控制和时间感知预验证控制，动态管理自适应草案算法执行和预验证时间，基于低置信度草案抑制无效草案。此外，AHASD在LPDDR5-PIM内集成了注意力算法单元和门控任务调度单元，以实现PIM侧的注意力链接定位和亚微秒级任务切换。针对不同LLM和自适应草案算法的实验结果表明，AHASD在GPU-only基准上实现了高达4.2倍的吞吐量和5.6倍的能效提升，并在GPU+PIM基准上实现了1.5倍的吞吐量和1.24倍的能效提升，硬件开销低于DRAM面积的3%。
英文摘要：
Speculative decoding enhances the inference efficiency of large language models (LLMs) by generating drafts using a small draft language model (DLM) and verifying them in batches with a large target language model (TLM). However, adaptive drafting inference on a mobile single-NPU-PIM system faces idle overhead in traditional operator-level synchronous execution and wasted computation in asynchronous execution due to fluctuations in draft length. This paper introduces AHASD, a task-level asynchronous mobile NPU-PIM heterogeneous architecture for speculative decoding. Notably, AHASD achieves parallel drafting on the PIM and verification on a single NPU through task-level DLM-TLM decoupling and specifically, it incorporates Entropy-History-Aware Drafting Control and Time-Aware Pre-Verification Control to dynamically manage adaptive drafting algorithm execution and pre-verification timing, suppressing invalid drafting based on low-confidence drafts. Additionally, AHASD integrates Attention Algorithm Units and Gated Task Scheduling Units within LPDDR5-PIM to enable attention link localization and sub-microsecond task switching on the PIM side. Experimental results for different LLMs and adaptive drafting algorithms show that AHASD achieves up to 4.2$\times$ in throughput and 5.6$\times$ in energy efficiency improvements over a GPU-only baseline, and 1.5$\times$ in throughput and 1.24$\times$ in energy efficiency gains over the state-of-the-art GPU+PIM baseline, with hardware overhead below 3% of the DRAM area.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
