能力越强，责任越大：降低语言模型训练能耗的建议
Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models
AI总结：
本文针对NLP模型能耗快速增长的问题，研究了能耗测量技术及可调的硬件、数据中心设置，经实验验证功率封顶等技术可在小幅增加计算时间的前提下显著降低语言模型训练与推理能耗。
摘要：
当前自然语言处理模型的能耗需求正以不可持续的速度快速增长。近期强调这一问题的研究得出结论：迫切需要能够降低NLP及更广泛的机器学习领域能耗需求的方法。本文研究了可用于降低常见NLP应用能耗的技术。我们重点关注能耗测量技术，以及可通过调整来降低语言模型训练与推理能耗的不同硬件配置和面向数据中心的设置。我们通过在高性能计算系统以及主流云计算平台上开展实验，明确了这些设置对计算性能、能耗等指标的影响。这些技术可大幅降低语言模型训练或推理使用时的能耗。例如，功率封顶（限制GPU可消耗的最大功率）技术在训练基于transformer的语言模型时，可实现15%的能耗降低，且整体计算时间仅略有增加。
英文摘要：
The energy requirements of current natural language processing models continue to grow at a rapid, unsustainable pace. Recent works highlighting this problem conclude there is an urgent need for methods that reduce the energy needs of NLP and machine learning more broadly. In this article, we investigate techniques that can be used to reduce the energy consumption of common NLP applications. In particular, we focus on techniques to measure energy usage and different hardware and datacenter-oriented settings that can be tuned to reduce energy consumption for training and inference for language models. We characterize the impact of these settings on metrics such as computational performance and energy consumption through experiments conducted on a high performance computing system as well as popular cloud computing platforms. These techniques can lead to significant reduction in energy consumption when training language models or their use for inference. For example, power-capping, which limits the maximum power a GPU can consume, can enable a 15\% decrease in energy usage with marginal increase in overall computation time when training a transformer-based language model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
