发表机构
Duke University（杜克大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出DynaCore架构，通过重塑MEU和解聚量化协同设计，显著提升LLM服务效率，TTFT提升3.50倍，TPOT提升36.55倍。
AI 中文摘要
大型语言模型（LLMs）已成为现代AI应用的支柱，但对高效推理提出了重大挑战。其自回归生成将执行分为两个阶段：预填充阶段，由大型GEMM主导；解码阶段，由小型GEMV主导。现代服务系统通过连续批处理和预填充-解码解聚进一步引入复杂性，导致动态工作负载和阶段分离。然而，现有加速器与这些系统级行为仍不匹配，导致LLM服务效率低下。在本工作中，我们提出了DynaCore，一种通过系统-架构协同设计实现高效LLM服务的统一架构。我们观察到，脉动阵列执行的计算块，即其最小高效单元（MEU），跨越所有三个GEMM维度。DynaCore沿三个维度重塑MEU：在空间上，它非对称地权衡阵列宽度与高度，提高权重传输而保持输入路径不变；在时间上，Split-K将归约映射到阵列上，通过阵列已有的互连折叠部分和。为利用阶段分离，我们进一步提出解聚量化，对预填充应用双重量化，对解码应用仅权重量化，并采用内积混合精度数据路径，使输出宽度与精度无关。运行时调度框架随后为每个批次选择MEU。使用真实服务轨迹的评估表明，DynaCore在量化与可重构加速器上大幅降低服务级延迟，分别将TTFT提升3.50倍和2.97倍，将TPOT提升36.55倍和8.02倍。
英文摘要
Large language models (LLMs) have become the backbone of modern AI applications, but pose significant challenges for efficient inference. Their autoregressive generation divides execution into two phases: prefill, dominated by large GEMMs, and decoding, dominated by small GEMVs. Modern serving systems further introduce complexity through continuous batching and prefill-decoding disaggregation, leading to dynamic workloads and phase separation. However, existing accelerators remain poorly aligned with these system-level behaviors, resulting in inefficiencies in LLM serving. In this work, we present DynaCore, a unified architecture for efficient LLM serving via system-architecture co-design. We observe that the compute tile a systolic array executes, its Minimum Efficient Unit (MEU), spans all three GEMM dimensions. DynaCore reshapes the MEU along all three: spatially it trades array width against height asymmetrically, raising weight delivery while leaving the input path untouched, and temporally Split-K maps the reduction onto the array, folding partial sums through the interconnect the array already has. To exploit phase separation, we further propose disaggregated quantization, applying dual-side quantization to prefill and weight-only quantization to decoding, with an inner-product mixed-precision datapath that keeps output width invariant to precision. A runtime scheduling framework then selects an MEU per batch. Evaluation with real-world serving traces shows that DynaCore substantially reduces service-level latency over quantization and reconfigurable accelerators, improving TTFT by 3.50x and 2.97x and TPOT by 36.55x and 8.02x, respectively.
Comments14 pages, 14 figures, 4 tables
