arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.01625cs.LG
收藏

NVIDIA边缘板卡上神经网络的能耗:一个经验模型

Energy Consumption of Neural Networks on NVIDIA Edge Boards: an Empirical Model

University of Padova(帕多瓦大学)

Seyyidahmed Lahmer, Aria Khoshsirat, Michele Rossi, Andrea Zanella

AI总结:

本文通过在NVIDIA Jetson TX2和Xavier上实测卷积层与全连接层推理能耗,提出一个可估计边缘神经网络能耗的实用经验模型。

摘要:

近年来,出现了将深度学习推理任务的执行转移到网络边缘、更靠近用户的趋势,以降低延迟并保护数据隐私。与此同时,机器学习的能源可持续性正受到越来越多的关注。因此,在这些趋势的交汇处,我们看到边缘端机器学习的能耗特性刻画正吸引日益增长的关注。遗憾的是,由于可能的底层硬件实现具有异构性,计算给定神经网络在推理期间的能耗十分复杂。因此,在本工作中,我们旨在对一些现代边缘节点的推理任务能耗进行性能分析,并推导简单但符合实际的模型。为此,我们开展了大量实验,在两款知名的NVIDIA边缘板卡(即Jetson TX2和Xavier)上采集卷积层和全连接层的能耗。随后,我们从测量结果中提炼出一个简单、实用的模型,该模型能够估计特定推理任务在所考虑板卡上的能耗。我们相信,该模型可用于多种场景,例如在Neural Architecture Search(神经架构搜索)中指导高效架构的搜索,在Neural Network pruning(神经网络剪枝)中作为启发式方法,或在Split computing(分割计算)场景中寻找高能效的卸载策略,也可简单用于评估Deep Neural Network架构的能耗性能。

英文摘要:

Recently, there has been a trend of shifting the execution of deep learning inference tasks toward the edge of the network, closer to the user, to reduce latency and preserve data privacy. At the same time, growing interest is being devoted to the energetic sustainability of machine learning. At the intersection of these trends, we hence find the energetic characterization of machine learning at the edge, which is attracting increasing attention. Unfortunately, calculating the energy consumption of a given neural network during inference is complicated by the heterogeneity of the possible underlying hardware implementation. In this work, we hence aim at profiling the energetic consumption of inference tasks for some modern edge nodes and deriving simple but realistic models. To this end, we performed a large number of experiments to collect the energy consumption of convolutional and fully connected layers on two well-known edge boards by NVIDIA, namely Jetson TX2 and Xavier. From the measurements, we have then distilled a simple, practical model that can provide an estimate of the energy consumption of a certain inference task on the considered boards. We believe that this model can be used in many contexts as, for instance, to guide the search for efficient architectures in Neural Architecture Search, as a heuristic in Neural Network pruning, or to find energy-efficient offloading strategies in a Split computing context, or simply to evaluate the energetic performance of Deep Neural Network architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑