持续学习究竟有多绿色？分析视觉基础模型持续训练中的能耗
How green is continual learning, really? Analyzing the energy consumption in continual training of vision foundation models
AI总结：
本文通过大量实验系统比较了多种持续学习算法在视觉基础模型持续训练中的能耗，并提出Energy NetScore指标，揭示推理阶段能耗对绿色评估的重要性。
摘要：
随着人工智能的日益普及，其对环境的影响已不容忽视。尽管持续学习在绿色人工智能方面具有潜力，但其环境可持续性仍然相对未被探索。本研究旨在系统理解持续学习算法的能源效率。为此，我们进行了大量实证实验，比较了近期基于表示、提示和样本的持续学习算法以及两种标准基线（微调和联合训练）在持续适配预训练ViT-B/16基础模型时的能耗。我们在三个标准数据集上进行了实验：CIFAR-100、ImageNet-R和DomainNet。此外，我们提出了一种新指标——Energy NetScore，用于衡量算法在能量-精度权衡方面的效率。通过改变增量学习步骤的数量和规模进行大量评估，我们的实验表明，不同类型的持续学习算法在训练和推理阶段的能耗影响差异很大。尽管在持续学习文献中常被忽视，我们发现推理阶段消耗的能量对于评估持续学习模型的环境可持续性至关重要。
英文摘要：
With the ever-growing adoption of AI, its impact on the environment is no longer negligible. Despite the potential that continual learning could have towards Green AI, its environmental sustainability remains relatively uncharted. In this work we aim to gain a systematic understanding of the energy efficiency of continual learning algorithms. To that end, we conducted an extensive set of empirical experiments comparing the energy consumption of recent representation-, prompt-, and exemplar-based continual learning algorithms and two standard baseline (fine tuning and joint training) when used to continually adapt a pre-trained ViT-B/16 foundation model. We performed our experiments on three standard datasets: CIFAR-100, ImageNet-R, and DomainNet. Additionally, we propose a novel metric, the Energy NetScore, which we use measure the algorithm efficiency in terms of energy-accuracy trade-off. Through numerous evaluations varying the number and size of the incremental learning steps, our experiments demonstrate that different types of continual learning algorithms have very different impacts on energy consumption during both training and inference. Although often overlooked in the continual learning literature, we found that the energy consumed during the inference phase is crucial for evaluating the environmental sustainability of continual learning models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
