arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.01617cs.AR
收藏

3DLS:一种用于分离式LLM服务的3D逻辑堆叠架构

3DLS: A 3D Logic-Stacked Architecture for Disaggregated LLM Serving

KAIST(韩国科学技术院)

Jaehun Lee, In-Jun Jung, Joo-Young Kim

AI总结:

针对预填-解码分离与张量并行中KV缓存传输和TP集合共享互连导致的关键路径争用问题,提出3D逻辑堆叠架构3DLS,通过垂直互连分离流量,提升吞吐量达1.49倍,降低端到端延迟60.2%。

摘要:

大语言模型(LLM)服务日益将预填-解码(PD)分离与张量并行(TP)相结合,以支持大模型和长上下文。在传统的2D/2.5D芯片架构中,逐层的预填到解码KV缓存传输和解码侧TP集合共享相同的横向芯片到芯片(D2D)互连,在解码关键路径上产生混合流量争用。这种争用增加了通信延迟,延长了令牌生成间隔,并降低了端到端服务性能。我们提出3DLS,一种逻辑上堆叠的3D芯片架构,通过垂直互连路由KV缓存传输来分离流量类别,同时将解码侧TP集合保留在横向D2D结构上。与共享结构的平面基线相比,3DLS实现了高达1.49倍的吞吐量和60.2%的端到端(E2E)延迟降低;与考虑工作负载的优先级管理平面基线相比,仍实现了高达1.17倍的吞吐量和31.4%的E2E延迟降低。这些结果突显了物理隔离是未来基于芯片的PD分离LLM服务系统的重要设计原则。

英文摘要:

Large language model (LLM) serving increasingly combines prefill-decode (PD) disaggregation with tensor parallelism (TP) to support large models and long contexts. In conventional 2D/2.5D chiplet architectures, layer-wise prefill-to-decode KV-cache transfer decode-side TP collectives share the same lateral die-to-die (D2D) interconnect, creating mixed-traffic contention on the decode critical path. This contention increases communication latency, prolongs token generation intervals, and degrades end-to-end serving performance. We propose 3DLS, a logic-on-logic 3D-stacked chiplet architecture that separates traffic classes by routing KV-cache transfers through vertical interconnects while preserving decode-side TP collectives on the lateral D2D fabric. 3DLS achieves up to 1.49$\times$ throughput and 60.2\% lower end-to-end (E2E) latency over the shared-fabric planar baseline, and still achieves up to 1.17$\times$ throughput and 31.4\% lower E2E latency over a workload-aware priority-managed planar baseline. These results highlight that physical isolation is an important design principle for future chiplet-based PD-disaggregated LLM serving systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑