PIMphony: 逾越基于PIM的长上下文LLM推理系统的带宽和容量低效问题
PIMphony: Overcoming Bandwidth and Capacity Inefficiency in PIM-based Long-Context LLM Inference System
AI总结:
PIMphony通过三种协同设计技术解决基于PIM的长上下文LLM推理系统的带宽和容量低效问题,显著提升吞吐量和效率。
摘要:
长上下文大型语言模型(LLM)的扩展带来了显著的内存系统挑战。虽然处理在内存(PIM)是一种有前途的加速器,但我们发现当扩展到长上下文时,它会面临关键的低效问题:严重的通道利用率不足、限制性能的I/O瓶颈以及大量内存浪费来自静态KV缓存管理。在本工作中,我们提出了PIMphony,一种PIM协调器,通过三种协同设计的技术系统地解决这些问题。首先,基于标记的PIM分区(TCP)确保无论批大小如何都能实现高通道利用率。其次,动态PIM命令调度(DCS)通过重叠数据移动和计算来缓解I/O瓶颈。最后,动态PIM访问(DPA)控制器能够实现动态内存管理,以消除静态内存浪费。通过基于MLIR的编译器实现,并在精确周期模拟器上进行评估,PIMphony显著提高了长上下文LLM推理的吞吐量(最多72B参数和1M上下文长度)。我们的评估显示在仅PIM系统上性能提升最高可达11.3倍,在xPU+PIM系统上性能提升最高可达8.4倍,使LLM在现实世界长上下文应用中的部署更加高效。
英文摘要:
The expansion of long-context Large Language Models (LLMs) creates significant memory system challenges. While Processing-in-Memory (PIM) is a promising accelerator, we identify that it suffers from critical inefficiencies when scaled to long contexts: severe channel underutilization, performance-limiting I/O bottlenecks, and massive memory waste from static KV cache management. In this work, we propose PIMphony, a PIM orchestrator that systematically resolves these issues with three co-designed techniques. First, Token-Centric PIM Partitioning (TCP) ensures high channel utilization regardless of batch size. Second, Dynamic PIM Command Scheduling (DCS) mitigates the I/O bottleneck by overlapping data movement and computation. Finally, a Dynamic PIM Access (DPA) controller enables dynamic memory management to eliminate static memory waste. Implemented via an MLIR-based compiler and evaluated on a cycle-accurate simulator, PIMphony significantly improves throughput for long-context LLM inference (up to 72B parameters and 1M context length). Our evaluations show performance boosts of up to 11.3x on PIM-only systems and 8.4x on xPU+PIM systems, enabling more efficient deployment of LLMs in real-world long-context applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。