Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. We present IrEne, an interpretable and extensible energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models. IrEne constructs a model tree graph that breaks down the NLP model into modules that are further broken down into low-level machine learning (ML) primitives. IrEne predicts the inference energy consumption of the ML primitives as a function of generalizable features and fine-grained runtime resource usage. IrEne then aggregates these low-level predictions recursively to predict the energy of each module and finally of the entire model. Experiments across multiple Transformer models show IrEne predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. In contrast, existing energy models see an error of over 50%. We also show how IrEne can be used to conduct energy bottleneck analysis and to easily evaluate the energy impact of different architectural choices. We release the code and data at https://github.com/StonyBrookNLP/irene.
Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoning
Xi’an Jiaotong University(西安交通大学);Shaanxi Province Key Laboratory of Satellite and Terrestrial Network Tech. R&D(陕西省卫星与地面网络技术研发重点实验室);National Engineering lab for Big Data Analytics(国家大数据分析工程实验室)
A hybrid transformer and attention based recurrent neural network for robust and interpretable sentiment analysis of tweets
Khulna University of Engineering & Technology (KUET)(库尔纳工程技术大学);Advanced Machine Intelligence Research (AMIR) Lab(高级机器智能研究实验室);American International University-Bangladesh(孟加拉国美国国际大学);Chowagiken Corp.(Chowagiken公司);University of Asia Pacific(亚太大学);University of Aizu(会津大学)
LAG-XAI: A Lie-Inspired Affine Geometric Framework for Interpretable Paraphrasing in Transformer Latent Spaces
Khmelnytskyi National University(赫梅利尼茨基国立大学);Taras Shevchenko National University of Kyiv(基辅塔拉斯·舍甫琴科国立大学);V.M. Glushkov Institute of Cybernetics of NAS of Ukraine(乌克兰国家科学院V.M.格卢什科夫控制论研究所)
Interpretable AI for Time-Series: Multi-Model Heatmap Fusion with Global Attention and NLP-Generated Explanations
Department of Electrical and Computer Engineering, Iowa State University, Ames, 50011 IA USA(电气与计算机工程系,爱荷华州立大学);Department of Agricultural Biosystem Engineering, Iowa State University, Ames,IA 50011 USA(农业生物系统工程系,爱荷华州立大学)
Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);School of Artificial Intelligence for Science, Peking University(北京大学科学人工智能学院)
DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning
National University of Singapore(新加坡国立大学);Singapore-MIT Alliance for Research and Technology(新加坡-麻省理工学院研究与技术联盟);Institute for Infocomm Research, A*STAR(新加坡科技研究局信息通信研究所);CSAIL, MIT(麻省理工学院计算机科学与人工智能实验室)