arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.13640cs.SEcs.LG
收藏

绿色AI:跨不同运行时基础设施的深度学习模型能耗初步实证研究

Green AI: A Preliminary Empirical Study on Energy Consumption in DL Models Across Different Runtime Infrastructures

Utrecht University(乌得勒支大学);University of Twente(特文特大学);Universidade Federal de Pernambuco(伯南布哥联邦大学)

Negar Alizadeh, Fernando Castor

AI总结:

本文实证测量PyTorch、TensorFlow、MXNet及ONNX运行时在三类DL模型上的能耗与推理时间,发现能效随框架、批大小和执行提供程序显著变化,且ONNX与TensorRT通常带来性能提升。

摘要:

PyTorch和TensorFlow等深度学习(DL)框架包含运行时基础设施,负责在目标硬件上执行已训练模型,并管理内存、数据传输以及多加速器执行(如适用)。此外,将预训练模型部署在与其原生开发环境不同的环境中是一种常见做法。这促成了ONNX等交换格式的引入,ONNX包含自身的运行时基础设施,而ONNX Runtime则作为可跨多种DL框架和语言使用的标准格式。尽管这些运行时基础设施对推理性能有很大影响,但此前没有论文研究过它们的能效。在本研究中,我们使用三种不同的DL模型,监测三个知名DL框架以及ONNX的运行时基础设施的能耗和推理时间。为使研究更细致,我们还考察了使用不同执行提供程序的影响。我们发现DL的性能和能效很难预测。在批大小为1时,由于高效的GPU使用以及由此带来的低CPU占用,MXNet这一框架在计算机视觉模型上的表现优于PyTorch和TensorFlow。然而,批大小为64时,PyTorch和MXNet实际上难以区分,而TensorFlow始终表现较差。对于BERT,PyTorch表现出最佳性能。在大多数情况下,将模型转换为ONNX可带来显著的性能提升。最后,在对执行提供程序的初步调查中,我们观察到TensorRT始终优于CUDA。

英文摘要:

Deep Learning (DL) frameworks such as PyTorch and TensorFlow include runtime infrastructures responsible for executing trained models on target hardware, managing memory, data transfers, and multi-accelerator execution, if applicable. Additionally, it is a common practice to deploy pre-trained models on environments distinct from their native development settings. This led to the introduction of interchange formats such as ONNX, which includes its runtime infrastructure, and ONNX Runtime, which work as standard formats that can be used across diverse DL frameworks and languages. Even though these runtime infrastructures have a great impact on inference performance, no previous paper has investigated their energy efficiency. In this study, we monitor the energy consumption and inference time in the runtime infrastructures of three well-known DL frameworks as well as ONNX, using three various DL models. To have nuance in our investigation, we also examine the impact of using different execution providers. We find out that the performance and energy efficiency of DL are difficult to predict. One framework, MXNet, outperforms both PyTorch and TensorFlow for the computer vision models using batch size 1, due to efficient GPU usage and thus low CPU usage. However, batch size 64 makes PyTorch and MXNet practically indistinguishable, while TensorFlow is outperformed consistently. For BERT, PyTorch exhibits the best performance. Converting the models to ONNX yields significant performance improvements in the majority of cases. Finally, in our preliminary investigation of execution providers, we observe that TensorRT always outperforms CUDA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑