As various post hoc explanation methods are increasingly being leveraged to explain complex models in high-stakes settings, it becomes critical to develop a deeper understanding of whether and when the explanations output by these methods disagree with each other, and how such disagreements are resolved in practice. However, there is little to no research that provides answers to these critical questions. In this work, we formalize and study the disagreement problem in explainable machine learning. More specifically, we define the notion of disagreement between explanations, analyze how often such disagreements occur in practice, and how practitioners resolve these disagreements. We first conduct interviews with data scientists to understand what constitutes disagreement between explanations generated by different methods for the same model prediction, and introduce a novel quantitative framework to formalize this understanding. We then leverage this framework to carry out a rigorous empirical analysis with four real-world datasets, six state-of-the-art post hoc explanation methods, and six different predictive models, to measure the extent of disagreement between the explanations generated by various popular explanation methods. In addition, we carry out an online user study with data scientists to understand how they resolve the aforementioned disagreements. Our results indicate that (1) state-of-the-art explanation methods often disagree in terms of the explanations they output, and (2) machine learning practitioners often employ ad hoc heuristics when resolving such disagreements. These findings suggest that practitioners may be relying on misleading explanations when making consequential decisions. They also underscore the importance of developing principled frameworks for effectively evaluating and comparing explanations output by various explanation techniques.
Detection of Disengagement from Voluntary Quizzes: An Explainable Machine Learning Approach in Higher Distance Education
Institute for Research in Open-, Distance- and eLearning, Swiss Distance University of Applied Sciences（开放、远程和e学习研究所，瑞士应用远程大学）；Department of Informatics, Swiss Distance University of Applied Sciences（信息学院，瑞士应用远程大学）；North-West University（北开普省大学）
An Explainable AI based approach for Monitoring Animal Health
Water Technology Development Hub(AWaDH), Indian Institute of Technology（印度理工学院水技术发展中心）；Academy of Scientific and Innovative Research (AcSIR)（科学与创新研究院）；Department of Higher Education（高等教育部）
Explainable Machine Learning using Real, Synthetic and Augmented Fire Tests to Predict Fire Resistance and Spalling of RC Columns
Clemson University（克莱姆森大学）；AI Research Institute for Science and Engineering (AIRISE), Clemson University（克莱姆森大学人工智能科学与工程研究院）；Michigan State University（密歇根州立大学）
An Explainable Machine Learning Approach to Traffic Accident Fatality Prediction
Institute of Information Technology（信息技术学院）；University of Dhaka（达卡大学）；Accident Research Institute（事故研究所）；Bangladesh University of Engineering and Technology（孟加拉工程技术大学）
The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus
Technical University of Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；UiT Arctic University of Norway（挪威北极大学）；Fraunhofer Heinrich-Hertz-Institute（弗劳恩霍夫海因里希·赫兹研究所）；University of Potsdam（波茨坦大学）；Leibniz Institute of Agricultural Engineering and Bioeconomy e.V.（莱布尼茨农业工程与生物经济研究所）
The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
SMT-EX: An Explainable Surrogate Modeling Toolbox for Mixed-Variables Design Exploration
AwanTunai（阿旺图奈）；ONERA（法国航空航天研究院）；Université de Toulouse（图卢兹大学）；Bandung Institute of Technology（万隆理工学院）；ISAE-SUPAERO（法国高等航空航天学院）；MINES ALBI（阿尔比矿业学院）；INSA（法国国立应用科学学院）；CNRS（法国国家科学研究中心）
Enabling Machine Learning Algorithms for Credit Scoring -- Explainable Artificial Intelligence (XAI) methods for clear understanding complex predictive models
Warsaw University of Technology（华沙理工大学）；University of Warsaw（华沙大学）；Data Juice Lab sp. z o.o.（Data Juice Lab 有限责任公司）；Polish Credit Bureau S. A. (Biuro Informacji Kredytowej S.A.)（波兰信贷局）
From Attribution to Action: A Human-Centered Application of Activation Steering
Fraunhofer Heinrich-Hertz-Institut（弗劳恩霍夫 Heinrich-Hertz 研究所）；Technische Universität Berlin（柏林技术大学）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
Am I Being Treated Fairly? A Conceptual Framework for Individuals to Ascertain Fairness
Data Science and Computational Intelligence Institute (DaSCI), University of Granada（格拉纳达大学数据科学与计算智能研究所（DaSCI））；University of Bergen（卑尔根大学）；Department of Computer Science and Artificial Intelligence (DCSAI), University of Granada（格拉纳达大学计算机科学与人工智能系（DCSAI））
