arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.01990cs.LGcs.DC
收藏

迈向可持续的大型语言模型服务

Towards Sustainable Large Language Model Serving

University of Waterloo(滑铁卢大学);Purdue University(普渡大学)

Sophia Nguyen, Beihao Zhou, Yi Ding, Sihang Liu

AI总结:

本文从运营与隐含碳排放角度研究 LLM 服务,基于两类 GPU 对不同规模 LLaMA 进行性能和能耗表征,并结合电网碳强度、芯片面积与内存建模,揭示协同优化可持续服务系统的潜力。

摘要:

在本工作中,我们从碳排放视角研究 LLM,同时处理运营碳排放和隐含碳排放,并为可持续的 LLM 服务铺平道路。我们使用两种 Nvidia GPU(最新一代 RTX6000 Ada 和较旧一代 T4)表征具有 1B、3B 和 7B 参数的 LLaMA 的性能与能耗。我们基于能耗和来自三个电网区域(每个区域代表不同的能源结构)的碳强度,对运营碳排放进行解析建模,并基于芯片面积和内存大小对隐含碳排放进行建模。我们的表征和建模使我们深入理解 LLM 服务的性能、能耗和碳排放。研究结果凸显了通过同时考虑运营和隐含碳排放来优化可持续 LLM 服务系统的潜力。

英文摘要:

In this work, we study LLMs from a carbon emission perspective, addressing both operational and embodied emissions, and paving the way for sustainable LLM serving. We characterize the performance and energy of LLaMA with 1B, 3B, and 7B parameters using two Nvidia GPU types, a latest-generation RTX6000 Ada and an older-generation T4. We analytically model operational carbon emissions based on energy consumption and carbon intensities from three grid regions -- each representing a different energy source mix, and embodied carbon emissions based on chip area and memory size. Our characterization and modeling provide us with an in-depth understanding of the performance, energy, and carbon emissions of LLM serving. Our findings highlight the potential for optimizing sustainable LLM serving systems by considering both operational and embodied carbon emissions simultaneously.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑