arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.02666cs.AR
收藏

Chiplet Cloud:构建用于服务大型生成式语言模型的AI超级计算机

Chiplet Cloud: Building AI Supercomputers for Serving Large Generative Language Models

Huwan Peng, Scott Davidson, Richard Shi, Shuaiwen Leon Song, Michael Taylor

AI总结:

针对大语言模型服务成本高的问题,提出基于芯粒的ASIC超级计算机架构Chiplet Cloud,通过软硬件协同设计与可扩展片上内存系统优化单token总拥有成本,性能远超GPU/TPU云且支持更大规模稀疏模型。

摘要:

大型语言模型(LLM),例如OpenAI的ChatGPT和Google的Gemini,已在多项任务中展现出自回归AI模型前所未有的能力,在全球范围内引发了颠覆性技术创新。然而,随着模型规模持续增长,服务这些模型的成本也不断攀升,对LLM的普及构成了威胁。\n 为解决这一问题,我们提出Chiplet Cloud——一种基于芯粒(chiplet)的专用集成电路(ASIC)LLM超级计算机架构,其目标是优化每生成一个token的总拥有成本(TCO)。该架构是一种高度可参数化的ASIC与服务器级架构,借助数千个可复制的加速器模块协同工作,在云规模下扩展LLM的性能。为确定Chiplet Cloud架构的具体参数配置,我们实现了一种两阶段软硬件协同设计方法,该方法能够在海量设计空间中搜索,并基于精确的推理仿真,针对一系列LLM对架构进行微调。内存访问性能是LLM的常见瓶颈,因此我们推出了CC-MEM——一种适用于Chiplet Cloud架构的可扩展片上内存系统。借助CC-MEM,在内存访问的功耗与性能至关重要的设计场景中,Chiplet Cloud可仅使用静态随机存取存储器(SRAM)构建。CC-MEM还包含一个压缩解码器模块,通过“压缩存储、稠密加载”机制,在不影响计算单元的前提下支持稀疏模型。\n 我们在8种主流LLM上评估了Chiplet Cloud架构。使用经过微调的Chiplet Cloud服务器,其每token总拥有成本相比租用的GPU和TPU云分别提升97倍和18倍,相比自研的GPU和TPU云分别提升8.3倍和3.7倍。在稀疏度为60%的情况下,Chiplet Cloud还可支持1.7倍规模的更大模型。

英文摘要:

Large language models (LLMs) such as OpenAI's ChatGPT and Google's Gemini have demonstrated unprecedented capabilities of autoregressive AI models across multiple tasks triggering disruptive technology innovations around the world. However, as models continue to grow the cost to serve these models also continues to grow threatening the democratization of LLMs. To address this issue, we propose Chiplet Cloud, a chiplet-based ASIC LLM-supercomputer architecture whose goal is to optimize the total cost of ownership (TCO) per generated token. This architecture is a highly parameterizable ASIC and server-level architecture leveraging thousands of replicated accelerator modules collaborating to scale-up the performance of LLMs at cloud-scale. To determine specific parameterizations of the Chiplet Cloud architecture, we implemented a two-phase hardware-software co-design methodology that can search the massive design space and fine tune the architecture across a collection of LLMs based on an accurate inference simulation. A common bottleneck for LLMs is the memory access performance therefore we introduce CC-MEM, a scalable on-chip memory system for Chiplet Cloud architectures. Using the CC-MEM, Chiplet Clouds can be built using only SRAMs for design points where the power and performance of memory access is critical. The CC-MEM also includes a compression decoder module to add support for sparse models without impacting the compute units using a Store-as-Compressed, Load-as-Dense mechanism. We evaluate Chiplet Cloud architectures across eight popular LLMs. Using fine tuned Chiplet Cloud servers we are able to achieve $97\times$ and $18\times$ improvement in TCO/Token over rented GPU and TPU clouds, or a $8.3\times$ and $3.7\times$ improvement over fabricated GPU and TPU clouds respectively. Chiplet Cloud can also support $1.7\times$ larger models with a sparsity of 60\%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑