arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.00741cs.AIcs.ARcs.DC
收藏

DynamoLLM:面向性能与能源效率设计 LLM 推理集群

DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency

University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft Azure Research - Systems(微软Azure研究-系统)

Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas, Esha Choukse

AI总结:

本文提出 DynamoLLM 能源管理框架,通过依据工作负载动态调整实例、并行度与 GPU 频率,在满足延迟 SLO 的同时显著降低 LLM 推理的能耗、碳排放和成本。

摘要:

生成式大型语言模型(LLM)的快速演进和广泛采用,已使其成为各类应用中的关键工作负载。如今,LLM 推理集群接收大量具有严格服务水平目标(SLO)的查询。为达到所需性能,这些模型在高功耗 GPU 上执行,导致推理集群消耗大量能源,并因此产生过多碳排放。幸运的是,我们发现存在利用推理计算属性异构性和推理工作负载波动的重要机会,可显著提升能源效率。然而,这种多样且动态的环境形成了巨大的搜索空间,其中不同系统配置(例如实例数量、模型并行度和 GPU 频率)会转化为不同的能源-性能权衡。为应对这些挑战,我们提出 DynamoLLM,这是首个面向 LLM 推理环境的能源管理框架。DynamoLLM 自动且动态地重新配置推理集群,以在服务性能 SLO 下优化 LLM 服务的能源和成本。我们表明,在服务层面,DynamoLLM 可节约 53% 的能源和 38% 的运营碳排放,并为客户降低 61% 的成本,同时满足延迟 SLO。

英文摘要:

The rapid evolution and widespread adoption of generative large language models (LLMs) have made them a pivotal workload in various applications. Today, LLM inference clusters receive a large number of queries with strict Service Level Objectives (SLOs). To achieve the desired performance, these models execute on power-hungry GPUs causing the inference clusters to consume large amount of energy and, consequently, result in excessive carbon emissions. Fortunately, we find that there is a great opportunity to exploit the heterogeneity in inference compute properties and fluctuations in inference workloads, to significantly improve energy-efficiency. However, such a diverse and dynamic environment creates a large search-space where different system configurations (e.g., number of instances, model parallelism, and GPU frequency) translate into different energy-performance trade-offs. To address these challenges, we propose DynamoLLM, the first energy-management framework for LLM inference environments. DynamoLLM automatically and dynamically reconfigures the inference cluster to optimize for energy and cost of LLM serving under the service's performance SLOs. We show that at a service-level, DynamoLLM conserves 53% energy and 38% operational carbon emissions, and reduces 61% cost to the customer, while meeting the latency SLOs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑