Recent work uncovered potential flaws in \eg attribution or heatmap based saliency methods. A typical flaw is a confirmations bias, where the scores are compared to human expectation. Since measuring the quality of saliency methods is hard due to missing ground truth model reasoning, finding general limitations is also hard. This is further complicated, because masking-based evaluation on complex data can easily introduce a bias, as most methods cannot fully ignore inputs. In this work, we extend our previous analysis on the logical dataset framework ANDOR, where we showed that all analysed saliency methods fail to grasp all needed classification information for all possible scenarios. Specifically, this paper extends our previous work using analysis on more datasets, in order to better understand in which scenarios the saliency methods fail. Further, we apply the Global Coherence Representation as an additional evaluation method in order to enable actual input omission.
Saliency strikes back: How filtering out high frequencies improves white-box explanations
Carney Institute for Brain Science, Brown University(布朗大学卡尼脑科学研究所);CerCo, CNRS(法国国家科学研究中心CerCo研究所);SNCF(法国国家铁路公司);Institute of Mathematics of Toulouse, University Paul Sabatier(保罗萨巴蒂埃大学图卢兹数学研究所)
Human Interpretation of Saliency-based Explanation Over Text
Bosch Center for Artificial Intelligence(博世人工智能中心);University of Stuttgart(斯图加特大学);Bar Ilan University(巴伊兰大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所)
Challenging the Black Box: A Comprehensive Evaluation of Attribution Maps of CNN Applications in Agriculture and Forestry
Fraunhofer ITWM(弗劳恩霍夫信息技术与数学研究所);Thünen Institute of Wood Research(蒂森木材研究所);Institute of Machine Learning and Analysis (IMLA)(机器学习与分析研究所);Offenburg University(奥芬堡大学)
Attribution Explanations for Deep Neural Networks: A Theoretical Perspective
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);School of Cyber Science and Engineering, Xi’an Jiaotong University(西安交通大学网络安全工程学院);Department of Computer Science and Engineering, the John Hopcroft Center, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系、霍夫曼中心);Department of Data Science, New Jersey Institute of Technology(新泽西理工学院数据科学系)
Investigating the Effectiveness of Explainability Methods in Parkinson's Detection from Speech
University of Bologna(博洛尼亚大学);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会);Mila-Québec AI Institute(米拉-魁北克人工智能研究所);Concordia University(康考迪亚大学);Laval University(拉瓦尔大学);Université de Montréal(蒙特利尔大学)