arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.01157cs.CLcs.PF
收藏

绿色人工智能:探索大型语言模型训练中的碳足迹、缓解策略与权衡

Green AI: Exploring Carbon Footprints, Mitigation Strategies, and Trade Offs in Large Language Model Training

Columbia University(哥伦比亚大学);University of Chicago(芝加哥大学)

Vivian Liu, Yiqiao Yin

AI总结:

本文评估了大型语言模型训练的高碳足迹,提出减少碳排放的可持续训练措施,并对比不同GPU硬件对排放的影响,论证了在不牺牲性能的前提下训练更环保AI模型的可行性。

摘要:

自然语言处理领域的许多重要工作长期以来一直试图通过改进先前的模型训练方法、改变模型架构以及开发更深入的数据集来创造新的创新模型,以提升其性能。然而,随着自然语言处理领域的快速发展,温室气体排放也随之增加,引发了人们对训练大型语言模型(LLMs)所造成的环境损害的担忧。全面了解与人工智能相关的各种成本,尤其是与环境方面相关的成本,是确保人工智能模型安全的基础。目前,对人工智能模型二氧化碳排放的研究仍是一个新兴领域,因此,在本文中,我们评估了知名大型语言模型的二氧化碳排放情况,这些模型由于模型参数量巨大而具有特别高的碳足迹。我们主张以负责任和可持续的方式训练大型语言模型,并提出了减少碳排放的措施。此外,我们通过对比两种广泛使用的图形处理器(GPU)在模型训练过程中的二氧化碳排放,讨论了硬件选择如何影响二氧化碳排放。基于我们的研究结果,我们展示了所提出解决方案的优点和缺点,并论证了在不牺牲模型鲁棒性和性能的前提下训练更环保的人工智能模型的可能性。

英文摘要:

Prominent works in the field of Natural Language Processing have long attempted to create new innovative models by improving upon previous model training approaches, altering model architecture, and developing more in-depth datasets to better their performance. However, with the quickly advancing field of NLP comes increased greenhouse gas emissions, posing concerns over the environmental damage caused by training LLMs. Gaining a comprehensive understanding of the various costs, particularly those pertaining to environmental aspects, that are associated with artificial intelligence serves as the foundational basis for ensuring safe AI models. Currently, investigations into the CO2 emissions of AI models remain an emerging area of research, and as such, in this paper, we evaluate the CO2 emissions of well-known large language models, which have an especially high carbon footprint due to their significant amount of model parameters. We argue for the training of LLMs in a way that is responsible and sustainable by suggesting measures for reducing carbon emissions. Furthermore, we discuss how the choice of hardware affects CO2 emissions by contrasting the CO2 emissions during model training for two widely used GPUs. Based on our results, we present the benefits and drawbacks of our proposed solutions and make the argument for the possibility of training more environmentally safe AI models without sacrificing their robustness and performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑