arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.00368cs.LGcs.AIcs.CV
收藏

通过能量感知训练最小化深度学习模型的能耗

Minimizing Energy Consumption of Deep Learning Models by Energy-Aware Training

Ca’ Foscari University of Venice(威尼斯大学);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);University of Cagliari(卡利亚里大学);University of Genoa(热那亚大学)

Dario Lazzaro, Antonio Emanuele Cinà, Maura Pintor, Ambra Demontis, Battista Biggio, Fabio Roli, Marcello Pelillo

AI总结:

提出基于梯度的能量感知训练算法EAT,利用$\ell_0$范数可微近似作为稀疏惩罚,在三个数据集和两个深度神经网络上实现了分类性能与能效的更优权衡。

摘要:

深度学习模型的参数数量显著增加,导致推理期间执行更多的操作。这种扩张显著导致了更高的能耗和预测延迟。在本研究中,我们提出EAT,一种基于梯度的算法,旨在减少模型训练期间的能耗。为此,我们利用$\ell_0$范数的可微近似,并将其作为训练损失上的稀疏惩罚。通过在三个数据集和两个深度神经网络上进行的实验分析,我们证明我们的能量感知训练算法EAT能够训练出在分类性能和能效之间具有更好权衡的网络。

英文摘要:

Deep learning models undergo a significant increase in the number of parameters they possess, leading to the execution of a larger number of operations during inference. This expansion significantly contributes to higher energy consumption and prediction latency. In this work, we propose EAT, a gradient-based algorithm that aims to reduce energy consumption during model training. To this end, we leverage a differentiable approximation of the $\ell_0$ norm, and use it as a sparse penalty over the training loss. Through our experimental analysis conducted on three datasets and two deep neural networks, we demonstrate that our energy-aware training algorithm EAT is able to train networks with a better trade-off between classification performance and energy efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑