揭秘面向下一代 LLM 模型分布式推理的 AI 平台设计
Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models
AI总结：
本文提出 GenZ 分析工具，建模多种 LLM 架构、服务优化与硬件平台参数，经真实平台验证后揭示分布式推理的资源需求，以指导下一代 AI 加速器与平台设计。
摘要：
大语言模型（LLM）已在广泛应用中展现出卓越性能，往往超过人类专家。然而，要为多样化推理用例高效部署这些庞大模型，需要精心设计具备充足计算、内存和网络资源的硬件平台。随着 LLM 服务优化持续创新，且模型架构以极快速度演进，满足服务水平目标（SLO）所需的硬件要求仍是一个开放的研究问题。为回答该问题，我们提出一种分析工具 GenZ，用于高效梳理多样化 LLM 模型架构（Dense、GQA、MoE、Mamba）、LLM 服务优化（Chunking、Speculative decoding、量化）与 AI 平台设计参数之间的关系。该工具可针对给定场景估算 LLM 推理性能指标。我们已在运行多种不同 LLM 模型的真实硬件平台上进行验证，最大几何平均误差为 5.82。我们使用 GenZ 识别跨多样化 LLM 推理用例的计算、内存容量、内存带宽、网络延迟和网络带宽需求。我们还研究了当前使用的多种架构选择（灵感来自多家供应商的 LLM 服务平台），以帮助指导设计下一代 AI 硬件加速器和平台的计算机架构师。由 GenZ 得出的趋势和洞见可指导部署 LLM 的 AI 工程师，以及设计下一代硬件加速器和平台的计算机架构师。最终，本工作阐明了在各类应用中释放大语言模型全部潜力的平台设计考量。源代码位于 https://github.com/abhibambhaniya/GenZ-LLM-Analyzer 。用户也可在 https://genz-llm-analyzer.streamlit.app/ 上通过网页浏览器试用，无需任何设置。
英文摘要：
Large language models (LLMs) have shown remarkable performance across a wide range of applications, often outperforming human experts. However, deploying these gigantic models efficiently for diverse inference use cases requires carefully designed hardware platforms with ample computing, memory, and network resources. With constant innovation in LLM serving optimizations and model architecture evolving at breakneck speed, the hardware requirements to meet Service Level Objectives (SLOs) remain an open research question. To answer the question, we present an analytical tool, GenZ, to efficiently navigate the relationship between diverse LLM model architectures(Dense, GQA, MoE, Mamba), LLM serving optimizations(Chunking, Speculative decoding, quanitization), and AI platform design parameters. Our tool estimates LLM inference performance metrics for the given scenario. We have validated against real hardware platforms running various different LLM models, achieving a max geomean error of 5.82.We use GenZ to identify compute, memory capacity, memory bandwidth, network latency, and network bandwidth requirements across diverse LLM inference use cases. We also study diverse architectural choices in use today (inspired by LLM serving platforms from several vendors) to help inform computer architects designing next-generation AI hardware accelerators and platforms. The trends and insights derived from GenZ can guide AI engineers deploying LLMs as well as computer architects designing next-generation hardware accelerators and platforms. Ultimately, this work sheds light on the platform design considerations for unlocking the full potential of large language models across a spectrum of applications. The source code is available at https://github.com/abhibambhaniya/GenZ-LLM-Analyzer . Users can also be tried it on at https://genz-llm-analyzer.streamlit.app/ without any setup on your web browser.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
