在模型自适应全生命周期内估算环境成本
Estimating Environmental Cost Throughout Model's Adaptive Life Cycle
AI总结：
针对模型因数据分布偏移而再训练的能耗与碳排放问题，提出仅需一次前向传播的预测指标 PreIndex，用于估算再训练资源并支持可持续的模型复用决策。
摘要：
随着当今时代神经网络研究、开发和应用的快速增长，训练和使用模型所需的能源也成比例增加。关键在于，这还伴随着向环境中碳排放的增加。减少与现代 AI/深度学习时代相关的碳足迹和不断上升的能源需求的一种可持续且具有社会效益的方法，是根据模型部署环境的变化或输入数据的变化/改变，对模型进行自适应且持续的复用。本文提出 PreIndex，这是一种预测性指标，用于估算与针对数据分布偏移进行模型再训练相关的环境和计算资源。PreIndex 可用于估算从当前数据分布再训练到新数据分布时的碳排放和能源使用等环境成本。它还与深度学习相关的其他资源指标相关并可用于估算这些指标，例如 epoch、梯度范数和模型参数变化幅度。PreIndex 仅需对数据进行一次前向传播，随后给出一个简洁的单一数值，以估算与再训练到新的分布偏移数据相关的资源。我们表明，PreIndex 可以可靠地用于各种数据集、模型架构以及不同类型和强度的分布偏移。因此，PreIndex 使用户能够就是否针对不同分布偏移进行再训练作出知情决策，并确定最具成本效益和可持续性的选择，从而允许以小得多的环境足迹复用模型。本工作的代码可在以下地址获取：https://github.com/JEKimLab/AIES2024PreIndex
英文摘要：
With the rapid increase in the research, development, and application of neural networks in the current era, there is a proportional increase in the energy needed to train and use models. Crucially, this is accompanied by the increase in carbon emissions into the environment. A sustainable and socially beneficial approach to reducing the carbon footprint and rising energy demands associated with the modern age of AI/deep learning is the adaptive and continuous reuse of models with regard to changes in the environment of model deployment or variations/changes in the input data. In this paper, we propose PreIndex, a predictive index to estimate the environmental and compute resources associated with model retraining to distributional shifts in data. PreIndex can be used to estimate environmental costs such as carbon emissions and energy usage when retraining from current data distribution to new data distribution. It also correlates with and can be used to estimate other resource indicators associated with deep learning, such as epochs, gradient norm, and magnitude of model parameter change. PreIndex requires only one forward pass of the data, following which it provides a single concise value to estimate resources associated with retraining to the new distribution shifted data. We show that PreIndex can be reliably used across various datasets, model architectures, different types, and intensities of distribution shifts. Thus, PreIndex enables users to make informed decisions for retraining to different distribution shifts and determine the most cost-effective and sustainable option, allowing for the reuse of a model with a much smaller footprint in the environment. The code for this work is available here: https://github.com/JEKimLab/AIES2024PreIndex
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
