arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.07578cs.AR
收藏

PIM Is All You Need:一种基于CXL的无GPU大语言模型推理系统

PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inference

Yufeng Gu, Alireza Khadem, Sumanth Umesh, Ning Liang, Xavier Servot, Onur Mutlu, Ravi Iyer, Reetuparna Das

AI总结:

针对LLM推理中内存带宽受限及GPU计算资源利用率低的问题,提出基于CXL的无GPU系统CENT,利用CXL内存扩展和近存储体处理实现高带宽,在相似功耗下吞吐量提升2.3倍且能耗降低2.9倍,每美元生成的词元数量比GPU多5.2倍。

摘要:

大语言模型(LLM)推理采用自回归方式逐个生成词元,其操作强度明显低于早期的机器学习(ML)模型,如纯编码器transformer和卷积神经网络。同时,LLM具有庞大的参数规模,并使用键值缓存来存储上下文信息。现代LLM支持高达100万个词元的上下文窗口,以生成多样化的文本、音频和视频内容。每个提示词独有的庞大键值缓存需要巨大的内存容量,从而限制了推理批处理大小。较低的操作强度和受限的批处理大小共同要求极高的内存带宽。然而,当代用于ML模型部署的硬件系统(如GPU和TPU)主要针对计算吞吐量进行优化。这种不匹配挑战了高级LLM的高效部署,并导致用户为昂贵的计算资源付费,而这些资源在受限于内存的LLM推理任务中利用率很低。我们提出了CENT,一种CXL启用的无GPU的LLM推理系统,它利用CXL内存扩展功能来容纳庞大的LLM规模,并利用近存储体处理单元提供高内存带宽,从而消除了对昂贵GPU的需求。CENT采用可扩展的CXL网络来支持CXL设备间的点对点和集合通信原语。我们实现了各种并行策略,将LLM分布在这些设备上。与具有最大支持批处理大小和相似平均功耗的GPU基线相比,CENT实现了2.3倍的更高吞吐量,并消耗了2.9倍的更少能量。CENT提升了总拥有成本(TCO)效益,每美元生成的词元数量比GPU多5.2倍。

英文摘要:

Large Language Model (LLM) inference uses an autoregressive manner to generate one token at a time, which exhibits notably lower operational intensity compared to earlier Machine Learning (ML) models such as encoder-only transformers and Convolutional Neural Networks. At the same time, LLMs possess large parameter sizes and use key-value caches to store context information. Modern LLMs support context windows with up to 1 million tokens to generate versatile text, audio, and video content. A large key-value cache unique to each prompt requires a large memory capacity, limiting the inference batch size. Both low operational intensity and limited batch size necessitate a high memory bandwidth. However, contemporary hardware systems for ML model deployment, such as GPUs and TPUs, are primarily optimized for compute throughput. This mismatch challenges the efficient deployment of advanced LLMs and makes users pay for expensive compute resources that are poorly utilized for the memory-bound LLM inference tasks. We propose CENT, a CXL-ENabled GPU-Free sysTem for LLM inference, which harnesses CXL memory expansion capabilities to accommodate substantial LLM sizes, and utilizes near-bank processing units to deliver high memory bandwidth, eliminating the need for expensive GPUs. CENT exploits a scalable CXL network to support peer-to-peer and collective communication primitives across CXL devices. We implement various parallelism strategies to distribute LLMs across these devices. Compared to GPU baselines with maximum supported batch sizes and similar average power, CENT achieves 2.3$\times$ higher throughput and consumes 2.9$\times$ less energy. CENT enhances the Total Cost of Ownership (TCO), generating 5.2$\times$ more tokens per dollar than GPUs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑