WaferLLM：晶圆级的大语言模型推理
WaferLLM: Large Language Model Inference at Wafer Scale
AI总结：
本文提出首个晶圆级LLM推理系统WaferLLM，通过PLMR硬件模型及MeshGEMM/MeshGEMV算法优化数十万核心利用率，在Cerebras WSE2上实现较A100 GPU最高20倍推理加速与16倍能效提升。
摘要：
新兴的AI加速器日益采用晶圆级制造技术，在mesh架构中集成了数十万个AI核心，拥有大型分布式片上内存（总计数十GB）和超高片上内存带宽（数十PB/s）。然而，当前针对GPU等共享内存架构优化的LLM推理系统，无法充分利用这些加速器。我们引入了WaferLLM，这是首个晶圆级LLM推理系统。WaferLLM由新颖的PLMR模型指导，该模型捕捉了晶圆级架构独特的硬件特性。利用该模型，WaferLLM开创了晶圆级LLM并行策略，优化了数十万个片上核心的利用率。它还引入了MeshGEMM和MeshGEMV，这是首批旨在晶圆级加速器上有效扩展的GEMM和GEMV实现。评估表明，WaferLLM的加速器利用率比现有最佳方法高出200倍。借助晶圆级加速器（Cerebras WSE2），WaferLLM执行GEMV操作的速度比在NVIDIA A100 GPU上快606倍，能效高16倍。对于完整的LLM推理，与运行SGLang和vLLM的A100 GPU集群相比，WaferLLM实现了10-20倍的加速。随着晶圆级AI模型、软件和硬件不断成熟，这些优势预计将不断扩大。WaferLLM已在https://github.com/MeshInfra/WaferLLM开源。
英文摘要：
Emerging AI accelerators increasingly adopt wafer-scale manufacturing technologies, integrating hundreds of thousands of AI cores in a mesh architecture with large distributed on-chip memory (tens of GB in total) and ultra-high on-chip memory bandwidth (tens of PB/s). However, current LLM inference systems, optimized for shared memory architectures like GPUs, fail to exploit these accelerators fully. We introduce WaferLLM, the first wafer-scale LLM inference system. WaferLLM is guided by a novel PLMR model (pronounced as "Plummer") that captures the unique hardware characteristics of wafer-scale architectures. Leveraging this model, WaferLLM pioneers wafer-scale LLM parallelism, optimizing the utilization of hundreds of thousands of on-chip cores. It also introduces MeshGEMM and MeshGEMV, the first GEMM and GEMV implementations designed to scale effectively on wafer-scale accelerators. Evaluations show that WaferLLM achieves up to 200$\times$ higher accelerator utilization than state-of-the-art methods. Leveraging a wafer-scale accelerator (Cerebras WSE2), WaferLLM delivers GEMV operations 606$\times$ faster and 16$\times$ more energy-efficient than on an NVIDIA A100 GPU. For full LLM inference, WaferLLM achieves 10-20$\times$ speedups over A100 GPU clusters running SGLang and vLLM. These advantages are expected to grow as wafer-scale AI models, software, and hardware continue to mature. WaferLLM is open-sourced at https://github.com/MeshInfra/WaferLLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
