In recent years, the community of 'explainable artificial intelligence' (XAI) has created a vast body of methods to bridge a perceived gap between model 'complexity' and 'interpretability'. However, a concrete problem to be solved by XAI methods has not yet been formally stated. As a result, XAI methods are lacking theoretical and empirical evidence for the 'correctness' of their explanations, limiting their potential use for quality-control and transparency purposes. At the same time, Haufe et al. (2014) showed, using simple toy examples, that even standard interpretations of linear models can be highly misleading. Specifically, high importance may be attributed to so-called suppressor variables lacking any statistical relation to the prediction target. This behavior has been confirmed empirically for a large array of XAI methods in Wilming et al. (2022). Here, we go one step further by deriving analytical expressions for the behavior of a variety of popular XAI methods on a simple two-dimensional binary classification problem involving Gaussian class-conditional distributions. We show that the majority of the studied approaches will attribute non-zero importance to a non-class-related suppressor feature in the presence of correlated noise. This poses important limitations on the interpretations and conclusions that the outputs of these XAI methods can afford.
Scrutinizing XAI using linear ground-truth data with suppressor variables
Technische Universität Berlin(柏林工业大学);Eindhoven University of Technology(埃因霍温理工大学);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所(BIFOLD));Korea University(高丽大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Physikalisch-Technische Bundesanstalt(德国联邦物理技术研究院);Charité – Universitätsmedizin Berlin(柏林夏里特医学院)
Employing Explainable Artificial Intelligence (XAI) Methodologies to Analyze the Correlation between Input Variables and Tensile Strength in Additively Manufactured Samples
School of Industrial and Information Engineering, Politecnico Di Milano(米兰理工大学工业与信息工程学院);Symbiosis Institute of Technology, Symbiosis International (Deemed) University(Symbiosis International (Deemed) University Symbiosis Institute of Technology)
Minimizing False-Positive Attributions in Explanations of Non-Linear Models
Technical University of Denmark(丹麦技术大学);Technische Universität Berlin(柏林技术大学);Physikalisch-Technische Bundesanstalt, Berlin(柏林物理技术部);Charité – Universitätsmedizin Berlin(柏林查理医院大学医学中心)
Measuring Perceived Trust in XAI-Assisted Decision-Making by Eliciting a Mental Model
Eindhoven University of Technology(埃因霍温理工大学);Eindhoven Artificial Intelligence Systems Institute(埃因霍温人工智能系统研究所);Ca’ Foscari University of Venice(威尼斯大学)
Analysis and Evaluation of Explainable Artificial Intelligence on Suicide Risk Assessment
The University of Western Australia(西澳大利亚大学);Armadale Mental Health Service(阿玛代尔心理健康服务中心);Bethesda Clinic(贝塞斯达诊所);Co-HIVE Mental Health(Co-HIVE心理健康中心);Harry Perkins Institute of Medical Research(哈里·珀金斯医学研究所);Fiona Stanley Hospital(菲奥娜·斯坦利医院);University of Western Australia(西澳大利亚大学)
Exploring Energy Landscapes for Minimal Counterfactual Explanations: Applications in Cybersecurity and Beyond
Netcompany-Intrasoft S.A.(Netcompany-Intrasoft公司);Hellenic Mediterranean University(希腊地中海大学);Harokopio University of Athens(雅典哈罗科皮奥大学);University of Western Macedonia(西马其顿大学)
REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability
UiT The Arctic University of Norway(挪威北极大学);Technical University of Denmark(丹麦技术大学);Norwegian Computing Center(挪威计算中心);University of Copenhagen(哥本哈根大学)