arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.01199cs.CL
收藏

IrEne:可解释的Transformers能量预测

IrEne: Interpretable Energy Prediction for Transformers

Stony Brook University(石溪大学)

Qingqing Cao, Yash Kumar Lal, Harsh Trivedi, Aruna Balasubramanian, Niranjan Balasubramanian

摘要:

现有的基于软件的NLP模型的能量测量是不准确的,因为它们没有考虑能量消耗和模型执行之间的复杂相互作用。我们提出了IrEne,一个可解释和可扩展的能量预测系统,可以准确预测各种基于transformer的NLP模型的推理能耗。IrEne构建了一个模型树图,将NLP模型分解为模块,这些模块进一步分解为低级机器学习(ML)原语。IrEne将ML原语的推理能耗预测为可泛化特征和细粒度运行时资源使用的函数。然后,IrEne递归地聚合这些低级预测,以预测每个模块的能量,并最终预测整个模型的能量。跨多个Transformer模型的实验表明,IrEne预测Transformer模型的推理能耗,与地面实况相比,误差低于7%。相比之下,现有的能源模型的误差超过50%。我们还展示了IrEne如何用于进行能源瓶颈分析,并轻松评估不同建筑选择的能源影响。我们在https://github.com/StonyBrookNLP/irene上发布代码和数据。

英文摘要:

Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. We present IrEne, an interpretable and extensible energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models. IrEne constructs a model tree graph that breaks down the NLP model into modules that are further broken down into low-level machine learning (ML) primitives. IrEne predicts the inference energy consumption of the ML primitives as a function of generalizable features and fine-grained runtime resource usage. IrEne then aggregates these low-level predictions recursively to predict the energy of each module and finally of the entire model. Experiments across multiple Transformer models show IrEne predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. In contrast, existing energy models see an error of over 50%. We also show how IrEne can be used to conduct energy bottleneck analysis and to easily evaluate the energy impact of different architectural choices. We release the code and data at https://github.com/StonyBrookNLP/irene.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑