arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.06195cs.SE
收藏

调整超参数是否能推动绿色深度学习:对超参数与深度学习模型能耗相关性的实证研究

Can Adjusting Hyperparameters Lead to Green Deep Learning: An Empirical Study on Correlations between Hyperparameters and Energy Consumption of Deep Learning Models

State Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室)

Taoran Wang, Yanhui Li, Mingliang Ma, Lin Chen, Yuming Zhou

AI总结:

本文通过实证研究发现,调整超参数可降低深度学习模型能耗,且在并行环境中能耗变化更显著,表明超参数优化对绿色DL的重要性。

摘要:

背景:随着深度学习(DL)模型的发展,更大的数据集和更复杂的模型结构被应用,导致计算资源和能耗上升,这表明绿色DL模型应受到更多关注。目标:本文聚焦于一个新的视角来分析DL能耗:超参数对DL模型能耗的影响。方法:我们的方法涉及使用突变操作符来模拟从业者如何调整超参数,如周期和学习率。我们分别训练原始模型和突变模型,并收集能耗信息和运行时间性能指标。此外,我们还关注多个DL模型并行训练的场景。结果:为了检验超参数对能耗的影响,我们在五个现实世界DL模型上进行了广泛实验。结果表明:(1)许多研究的超参数与能耗有(正或负)相关性;(2)调整超参数可以使DL模型更环保,即在不损害性能的情况下减少能耗;(3)在并行环境中,能耗变得更加易变。结论:我们建议在开发DL模型时,超参数需要更多关注,因为适当调整超参数可以导致绿色DL模型。

英文摘要:

Context: Along with developing Deep learning (DL) models, larger datasets and more complex model structures are applied, leading to rising computing resources and energy consumption, which is an alert that green DL models should receive more attention. Objective: This paper focuses on a novel view to analyze DL energy consumption: the effect of hyperparameters on the energy cost of DL models. Method: Our approach involves using mutation operators to simulate how practitioners adjust hyperparameters, such as epochs and learning rates. We train the original and mutated models separately and gather energy information and run-time performance metrics. Moreover, we focus on the parallel scenario where multiple DL models are trained in parallel. Results: To examine the effect of hyperparameters on energy consumption, we conducted extensive experiments on five real-world DL models. The results show that (1) many hyperparameters studied have a (positive or negative) correlation with energy consumption, (2) adjusting hyperparameters can make DL models greener, i.e., lead to less energy consumption without performance damage, and (3) in a parallel environment, energy consumption becomes more susceptible to change. Conclusions: We suggest that hyperparameters need more attention in developing DL models, as appropriately adjusting hyperparameters would cause green DL models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑