Compute and Energy Consumption Trends in Deep Learning Inference
European Commission, Joint Research Centre(欧盟委员会联合研究中心)
Radosvet Desislavov, Fernando Martínez-Plumed, José Hernández-Orallo
备注:
For a revised version and its published version refer to: Desislavov, Radosvet, Fernando Martínez-Plumed, and José Hernández-Orallo. Trends in AI inference energy consumption: Beyond the performance-vs-parameter laws of deep learning. Sustainable Computing: Informatics and Systems, Volume 38, April 2023. (https://doi.org/10.1016/j.suscom.2023.100857)
发表信息:
"Trends in AI inference energy consumption: Beyond the performance-vs-parameter laws of deep learning" Sustainable Computing: Informatics and Systems (2023). Volume 38, April 2023, 100857
The progress of some AI paradigms such as deep learning is said to be linked to an exponential growth in the number of parameters. There are many studies corroborating these trends, but does this translate into an exponential increase in energy consumption? In order to answer this question we focus on inference costs rather than training costs, as the former account for most of the computing effort, solely because of the multiplicative factors. Also, apart from algorithmic innovations, we account for more specific and powerful hardware (leading to higher FLOPS) that is usually accompanied with important energy efficiency optimisations. We also move the focus from the first implementation of a breakthrough paper towards the consolidated version of the techniques one or two year later. Under this distinctive and comprehensive perspective, we study relevant models in the areas of computer vision and natural language processing: for a sustained increase in performance we see a much softer growth in energy consumption than previously anticipated. The only caveat is, yet again, the multiplicative factor, as future AI increases penetration and becomes more pervasive.
Trustworthy Graph Neural Networks: Aspects, Methods and Trends
Monash University(莫纳什大学);University of Melbourne(墨尔本大学);Griffith University(格里菲斯大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Duke University(杜克大学)
Using Google Trends as a proxy for occupant behavior to predict building energy consumption
National University of Singapore (NUS)(新加坡国立大学)
近年来,大量能源数据和先进的机器学习算法的可用性,使建筑能源预测研究激增。然而,能源预测模型中的变量之一,居住者行为,是预测性能的关键,但难以测量或耗时收集每个建筑物。这项研究提出了一种方法,利用搜索量的主题(例如,教育}或Microsoft Excel)作为居住者行为和建筑物使用的代理。首先检查线性相关性,以探索电表数据与Google趋势搜索词之间的关系,从而推断建筑物占用率。基于ASHRAE Great Energy Predictor III(GEPIII)竞赛数据集,对包含这些项的趋势前后的预测误差进行了比较和分析。结果表明,高度相关的Google趋势数据可以有效地将建筑物子集的整体RMSLE错误减少到GEPIII竞赛前五名获胜团队的表现水平。特别是,在公共假日和具有站点特定时间表的日子期间,RMSLE误差减少分别减少20-30%和2- 5%。这些结果显示了使用Google趋势通过自动识别特定场地和假日时间表来改善部分建筑存量的能源预测的潜力。
Compute Trends Across Three Eras of Machine Learning
University of Aberdeen(阿伯丁大学);Centre for the Governance of AI(人工智能治理中心);University of St Andrews(圣安德鲁斯大学);Massachusetts Institute of Technology(麻省理工学院);University of Tübingen(蒂宾根大学);Complutense University of Madrid(马德里康普顿斯大学)
The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model
Université Lumière Lyon 2(里昂第二大学);CNRS(法国国家科学研究中心);Ecole Centrale de Lyon(里昂中央理工学院);INSA Lyon(里昂国立应用科学学院);Universite Claude Bernard Lyon 1(克洛德·贝尔纳里昂第一大学);LIRIS, UMR5205(里昂信息图像与科学计算实验室,UMR5205)
Efficient Deep Learning for Biometrics: Overview, Challenges and Trends in Ear of Frugal AI
LIASD Laboratory University of Paris 8 Saint-Denis, France(LIASD实验室 巴黎第八大学圣丹尼斯分校);LIASD Laboratory National School of Artificial Intelligence Algiers, Algeria(LIASD实验室 国家人工智能学校阿尔吉尔);LSI/USTHB National School of Artificial Intelligence Algiers, Algeria(LSI/USTHB 国家人工智能学校阿尔吉尔);LIASD research Lab., University of Paris 8, France(1LIASD研究实验室 巴黎第八大学法国);National High School of Artificial Intelligence (ENSIA), Algiers, Algeria(2国家人工智能高级中学(ENSIA) 阿尔吉尔)
TRIZ Method for Urban Building Energy Optimization: GWO-SARIMA-LSTM Forecasting model
Purdue University(普渡大学);George Washington University(乔治·华盛顿大学);University of Pennsylvania(宾夕法尼亚大学);Georgia Institute of Technology(佐治亚理工学院);Koforidua Technical University(科福里杜亚技术大学)
Time Series Vector Autoregression Prediction of the Ecological Footprint based on Energy Parameters
Mathematical Institute of the S.A.S.A(塞尔维亚科学与艺术学院数学研究所);Technical Faculty in Bor, University of Belgrade(贝尔格莱德大学博尔技术学院);DIMES, University of Calabria(卡拉布里亚大学DIMES)