Motivated by distinct, though related, criteria, a growing number of attribution methods have been developed tointerprete deep learning. While each relies on the interpretability of the concept of "importance" and our ability to visualize patterns, explanations produced by the methods often differ. As a result, input attribution for vision models fail to provide any level of human understanding of model behaviour. In this work we expand the foundationsof human-understandable concepts with which attributionscan be interpreted beyond "importance" and its visualization; we incorporate the logical concepts of necessity andsufficiency, and the concept of proportionality. We definemetrics to represent these concepts as quantitative aspectsof an attribution. This allows us to compare attributionsproduced by different methods and interpret them in novelways: to what extent does this attribution (or this method)represent the necessity or sufficiency of the highlighted inputs, and to what extent is it proportional? We evaluate our measures on a collection of methods explaining convolutional neural networks (CNN) for image classification. We conclude that some attribution methods are more appropriate for interpretation in terms of necessity while others are in terms of sufficiency, while no method is always the most appropriate in terms of both.
Department of Computer & Information Science & Engineering, University of Florida(佛罗里达大学计算机与信息科学与工程系);School of Computer Science, University of Oklahoma(俄克拉荷马大学计算机科学学院);Department of Electrical Engineering and Computer Science, United States Military Academy(美国军事学院电子工程与计算机科学系)
A Characteristic Function for Shapley-Value-Based Attribution of Anomaly Scores
University of Applied Sciences and Arts Western Switzerland(瑞士西部应用科学与艺术大学);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心);Osaka University(大阪大学)
From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
Fraunhofer Heinrich-Hertz-Institute(弗劳恩霍夫海因里希-赫兹研究所);Technische Universität Berlin(柏林工业大学);BIFOLD – Berlin Institute for the Foundations of Learning and Data(BIFOLD – 柏林学习与数据基础研究所)
Explaining Time Series Classifiers with PHAR: Rule Extraction and Fusion from Post-hoc Attributions
Jagiellonian Human-Centered AI Lab, Mark Kac Center for Complex Systems Research(杰克利昂人类中心人工智能实验室、马克·卡复杂系统研究中心);Institute of Applied Computer Science, Jagiellonian University(应用计算机科学研究所、杰克利昂大学)