arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.02001cs.LG
收藏

176 B参数语言模型BLOOM的碳足迹估算

Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model

Hugging Face(Hugging Face公司);Graphcore(Graphcore公司);LISN(LISN实验室);ENSIIE(法国国立高等计算机、电子能源及机械自动化工程师学院)

Alexandra Sasha Luccioni, Sylvain Viguier, Anne-Laure Ligozat

摘要:

机器学习(ML)的进步带来了环境成本,因为训练ML模型需要大量的计算资源、能源和材料。在本文中,我们的目标是量化的碳足迹的BLOOM,176亿参数语言模型,在其生存周期。我们估计,如果我们只考虑动态功耗,BLOOM的最终培训排放约24.7吨的~ \carboneq ~,如果我们考虑从设备制造到基于能源的运营消耗的所有过程,则排放50.5吨。我们还研究了其部署的能源需求和碳排放,以通过实时接收用户查询的API端点进行推理。最后,我们讨论了精确估计ML模型的碳足迹的难度以及未来的研究方向,这些研究方向有助于改善碳排放报告。

英文摘要:

Progress in machine learning (ML) comes with a cost to the environment, given that training ML models requires significant computational resources, energy and materials. In the present article, we aim to quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM's final training emitted approximately 24.7 tonnes of~\carboneq~if we consider only the dynamic power consumption, and 50.5 tonnes if we account for all processes ranging from equipment manufacturing to energy-based operational consumption. We also study the energy requirements and carbon emissions of its deployment for inference via an API endpoint receiving user queries in real-time. We conclude with a discussion regarding the difficulty of precisely estimating the carbon footprint of ML models and future research directions that can contribute towards improving carbon emissions reporting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑