arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.05043cs.DC
收藏

EcoServe:设计具备碳感知能力的 AI 推理系统

EcoServe: Designing Carbon-Aware AI Inference Systems

Yueying Li, Zhanqiu Hu, Esha Choukse, Rodrigo Fonseca, G. Edward Suh, Udit Gupta

AI总结:

本文针对 LLM 推理的运营与隐含碳排放问题,基于生产轨迹观察提出碳感知资源供给与调度框架 EcoServe,并通过跨栈 ILP 设计实现最高 47% 的减排。

摘要:

LLM 的普及程度和规模快速增长,给计算基础设施带来了前所未有的需求。这些需求不仅消耗大量计算和内存资源,还消耗大量能源,从而产生巨大的运营和隐含碳排放。在本文中,我们基于对一家主要云服务提供商中两个生成式 AI 服务的生产部署进行建模并利用其轨迹,提出三项主要观察。第一,尽管 GPU 在运营碳排放中占主导地位,但主机处理系统(如 CPU、内存、存储)在隐含碳排放中占主导地位。第二,离线批处理推理占据了服务容量的很大一部分(最高达 55%)。第三,LLM 推理在硬件和工作负载之间存在不同层级的异构性。基于这些观察,我们设计了 EcoServe,这是一个面向 LLM 服务系统、具备碳感知能力的资源供给与调度框架。它基于四项原则——减少、复用、合理调配和回收(4R)。通过跨栈 ILP 建模与设计,我们证明,相较于以性能、能源和成本优化的设计点,EcoServe 可在维持性能目标和 SLO 的同时,将碳排放最高降低 47%。

英文摘要:

The rapid increase in LLM ubiquity and scale levies unprecedented demands on computing infrastructure. These demands not only incur large compute and memory resources but also significant energy, yielding large operational and embodied carbon emissions. In this work, we present three main observations based on modeling and traces from the production deployment of two Generative AI services in a major cloud service provider. First, while GPUs dominate operational carbon, host processing systems (e.g., CPUs, memory, storage) dominate embodied carbon. Second, offline, batch inference accounts for a significant portion (up to 55\%) of serving capacity. Third, there are different levels of heterogeneity across hardware and workloads for LLM inference. Based on these observations, we design EcoServe, a carbon-aware resource provision and scheduling framework for LLM serving systems. It is based on four principles - Reduce, Reuse, Rightsize, and Recycle (4R). With a cross-stack ILP formulation and design, we demonstrate that EcoServe can lower carbon emissions by up to 47\%, compared to performance, energy, and cost-optimized design points, while maintaining performance targets and SLOs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑