POLCA:大语言模型云提供商中的功率超额订阅
POLCA: Power Oversubscription in LLM Cloud Providers
AI总结:
针对LLM云数据中心功率瓶颈问题,提出稳健可靠的POLCA功率超额订阅框架,可在推理集群中多部署30%服务器且性能损失极小,提升数据中心功率效率与部署速度。
摘要:
大语言模型(LLM)领域的最新创新及其海量用例,迅速推高了数据中心GPU的计算容量需求。多家云提供商及其他企业已制定了大规模的数据中心扩容计划,以支撑这些新型工作负载。功率是数据中心的关键瓶颈资源之一,而随着LLM模型规模不断增大,其功耗也越来越高。本文表明,LLM集群中存在显著的功率超额订阅空间。功率超额订阅能够提升这类数据中心的功率效率,让单个数据中心可部署的服务器数量更多,同时还能缩短部署周期——因为新建数据中心的过程十分缓慢。\n我们对多种LLM及其不同配置的功耗模式进行了全面的特征分析,明确了推理与训练阶段功耗模式的差异。基于对这些LLM的分析,我们提出推断:LLM集群中推理工作负载的平均和峰值功率利用率不会很高。我们的推论与生产环境LLM集群的实际数据一致,数据显示推理工作负载为功率超额订阅提供了充足的余量。然而,在虚拟化环境中,GPU提供的遥测与控制手段十分有限,这使得构建可靠、稳健的功率超额订阅机制面临挑战。\n我们提出了POLCA——这是一款适用于GPU集群的功率超额订阅框架,具备稳健性、可靠性且易于部署。我们使用开源模型复现了生产环境中观测到的功耗模式,对POLCA进行了仿真实验,结果证明:在同一GPU推理集群中,我们可以多部署30%的服务器,且性能损失极小。
英文摘要:
Recent innovation in large language models (LLMs), and their myriad use-cases have rapidly driven up the compute capacity demand for datacenter GPUs. Several cloud providers and other enterprises have made substantial plans of growth in their datacenters to support these new workloads. One of the key bottleneck resources in datacenters is power, and given the increasing model sizes of LLMs, they are becoming increasingly power intensive. In this paper, we show that there is a significant opportunity to oversubscribe power in LLM clusters. Power oversubscription improves the power efficiency of these datacenters, allowing more deployable servers per datacenter, and reduces the deployment time, since building new datacenters is slow. We extensively characterize the power consumption patterns of a variety of LLMs and their configurations. We identify the differences between the inference and training power consumption patterns. Based on our analysis of these LLMs, we claim that the average and peak power utilization in LLM clusters for inference should not be very high. Our deductions align with the data from production LLM clusters, revealing that inference workloads offer substantial headroom for power oversubscription. However, the stringent set of telemetry and controls that GPUs offer in a virtualized environment, makes it challenging to have a reliable and robust power oversubscription mechanism. We propose POLCA, our framework for power oversubscription that is robust, reliable, and readily deployable for GPU clusters. Using open-source models to replicate the power patterns observed in production, we simulate POLCA and demonstrate that we can deploy 30% more servers in the same GPU cluster for inference, with minimal performance loss
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。