The ubiquity of machine learning based predictive models in modern society naturally leads people to ask how trustworthy those models are? In predictive modeling, it is quite common to induce a trade-off between accuracy and interpretability. For instance, doctors would like to know how effective some treatment will be for a patient or why the model suggested a particular medication for a patient exhibiting those symptoms? We acknowledge that the necessity for interpretability is a consequence of an incomplete formalisation of the problem, or more precisely of multiple meanings adhered to a particular concept. For certain problems, it is not enough to get the answer (what), the model also has to provide an explanation of how it came to that conclusion (why), because a correct prediction, only partially solves the original problem. In this article we extend existing categorisation of techniques to aid model interpretability and test this categorisation.
From Continuous Predictors to Clinical Thresholds: Early Evidence on Performance Trade-offs of Guideline-Based Categorisation for Ischaemic Stroke Outcome Prediction
Technological University Dublin（都柏林理工大学）；Aalborg University（奥尔堡大学）；Health Management Institute（健康管理研究所）；Trinity College Dublin（都柏林圣三一学院）
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Saarland University（萨尔大学）
This study proposes an innovative explainable predictive quality analytics solution to facilitate data-driven decision-making for process planning in manufacturing by combining process mining, machine learning, and explainable artificial intelligence (XAI) methods. For this purpose, after integrating the top-floor and shop-floor data obtained from various enterprise information systems, a deep learning model was applied to predict the process outcomes. Since this study aims to operationalize the delivered predictive insights by embedding them into decision-making processes, it is essential to …
Operationalizing Fairness: Post-Hoc Threshold Optimization Under Hard Resource Limits
Department of Computer Science and Engineering, DUIET, Dibrugarh University, Assam, India（迪布鲁大学计算机科学与工程系，DUIET，阿萨姆，印度）；Department of Computer Science and Engineering, MIT, Manipur University, 795003, India（曼尼普尔大学计算机科学与工程系，MIT，曼尼普尔大学，印度）
Self-explaining deep models are designed to learn the latent concept-based explanations implicitly during training, which eliminates the requirement of any post-hoc explanation generation technique. In this work, we propose one such model that appends an explanation generation module on top of any basic network and jointly trains the whole module that shows high predictive performance and generates meaningful explanations in terms of concepts. Our training strategy is suitable for unsupervised concept learning with much lesser parameter space requirements compared to baseline methods. Our prop…
Rutgers University（罗格斯大学）；Renmin University of China（中国人民大学）
State-of-the-art recommender systems have the ability to generate high-quality recommendations, but usually cannot provide intuitive explanations to humans due to the usage of black-box prediction models. The lack of transparency has highlighted the critical importance of improving the explainability of recommender systems. In this paper, we propose to extract causal rules from the user interaction history as post-hoc explanations for the black-box sequential recommendation mechanisms, whilst maintain the predictive accuracy of the recommendation model. Our approach firstly achieves counterfac…
Institute for Artificial Intelligence and Decision Support（人工智能与决策支持研究所）；Medical University of Vienna（维也纳医科大学）
Black-box Artificial Intelligence (AI) methods, e.g. deep neural networks, have been widely utilized to build predictive models that can extract complex relationships in a dataset and make predictions for new unseen data records. However, it is difficult to trust decisions made by such methods since their inner working and decision logic is hidden from the user. Explainable Artificial Intelligence (XAI) refers to systems that try to explain how a black-box AI model produces its outcomes. Post-hoc XAI methods approximate the behavior of a black-box by extracting relationships between feature va…
Q(D)O-ES: Population-based Quality (Diversity) Optimisation for Post Hoc Ensemble Selection in AutoML
University of Siegen（锡根大学）；LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML））；Leiden University（莱顿大学）；RWTH Aachen（亚琛工业大学）；University of British Columbia（不列颠哥伦比亚大学）
Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；University of Trento（特伦托大学）；Free University of Bozen Bolzano（博尔扎诺自由大学）
Recent papers have introduced a novel approach to explain why a Predictive Process Monitoring (PPM) model for outcome-oriented predictions provides wrong predictions. Moreover, they have shown how to exploit the explanations, obtained using state-of-the art post-hoc explainers, to identify the most common features that induce a predictor to make mistakes in a semi-automated way, and, in turn, to reduce the impact of those features and increase the accuracy of the predictive model. This work starts from the assumption that frequent control flow patterns in event logs may represent important fea…
Optimal decision making requires that classifiers produce uncertainty estimates consistent with their empirical accuracy. However, deep neural networks are often under- or over-confident in their predictions. Consequently, methods have been developed to improve the calibration of their predictive uncertainty both during training and post-hoc. In this work, we propose differentiable losses to improve calibration based on a soft (continuous) version of the binning operation underlying popular calibration-error estimators. When incorporated into training, these soft calibration losses achieve sta…
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；IBM Research（IBM研究院）
Bayesian decision theory provides an elegant framework for acting optimally under uncertainty when tractable posterior distributions are available. Modern Bayesian models, however, typically involve intractable posteriors that are approximated with, potentially crude, surrogates. This difficulty has engendered loss-calibrated techniques that aim to learn posterior approximations that favor high-utility decisions. In this paper, focusing on Bayesian neural networks, we develop methods for correcting approximate posterior predictive distributions encouraging them to prefer high-utility decisions…
Numerous algorithms have been proposed for detecting anomalies (outliers, novelties) in an unsupervised manner. Unfortunately, it is not trivial, in general, to understand why a given sample (record) is labelled as an anomaly and thus diagnose its root causes. We propose the following reduced-dimensionality, surrogate model approach to explain detector decisions: approximate the detection model with another one that employs only a small subset of features. Subsequently, samples can be visualized in this low-dimensionality space for human understanding. To this end, we develop PROTEUS, an AutoM…
Post-Hoc Uncertainty-Aware Explanations for Deployed Power Quality Disturbance Classifiers via Laplace Approximation
School of Engineering, Deakin University（德肯大学工程学院）；ARC Training Centre in Energy Technologies for Future Grids, School of Engineering, University of Wollongong（未来电网能源技术培训中心，沃尔灵宗大学工程学院）
Self-Explaining Neural Networks for Business Process Monitoring
School of Computer Science and Engineering, Hebrew University of Jerusalem（希伯来大学计算机科学与工程学院）；IBM Research（IBM研究院）；University of Piraeus（比雷埃夫斯大学）；Bar-Ilan University（巴伊兰大学）
Probabilistic predictions from neural networks which account for predictive uncertainty during classification is crucial in many real-world and high-impact decision making settings. However, in practice most datasets are trained on non-probabilistic neural networks which by default do not capture this inherent uncertainty. This well-known problem has led to the development of post-hoc calibration procedures, such as Platt scaling (logistic), isotonic and beta calibration, which transforms the scores into well calibrated empirical probabilities. A plausible alternative to the calibration approa…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Saarland University（萨尔大学）
The contemporary process-aware information systems possess the capabilities to record the activities generated during the process execution. To leverage these process specific fine-granular data, process mining has recently emerged as a promising research discipline. As an important branch of process mining, predictive business process management, pursues the objective to generate forward-looking, predictive insights to shape business processes. In this study, we propose a conceptual framework sought to establish and promote understanding of decision-making environment, underlying business pro…
Technical University of Munich（慕尼黑工业大学）；Helmholtz Munich（亥姆霍兹慕尼黑中心）；ELLIS Institute Finland（芬兰ELLIS研究所）；Aalto University（阿尔托大学）；Finnish Center for Artificial Intelligence（芬兰人工智能中心）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Munich Data Science Institute（慕尼黑数据科学研究所）；Ericsson Research（爱立信研究院）；KTH Royal Institute of Technology（KTH皇家理工学院）；Digital Futures（数字未来研究所）
Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyond
Columbia University（哥伦比亚大学）；Department of Computer Science, New York University（纽约大学计算机科学系）；Microsoft Research（微软研究院）；Center for Data Science, New York University（纽约大学数据科学中心）
To tackle interpretability in deep learning, we present a novel framework to jointly learn a predictive model and its associated interpretation model. The interpreter provides both local and global interpretability about the predictive model in terms of human-understandable high level attribute functions, with minimal loss of accuracy. This is achieved by a dedicated architecture and well chosen regularization penalties. We seek for a small-size dictionary of high level attribute functions that take as inputs the outputs of selected hidden layers and whose outputs feed a linear classifier. We …
Selecting Interpretability Techniques for Healthcare Machine Learning models
University of Antioquia UdeA（安蒂奥基亚大学）；Universidad de Antioquia UdeA（安蒂奥基亚大学）；University of Antioquia（安蒂奥基亚大学）；Hospital Alma Máter de Antioquia（安蒂奥基亚母校医院）；Arcada University of Applied Sciences（阿卡达应用科学大学）
Meta-learning is a field that aims at discovering how different machine learning algorithms perform on a wide range of predictive tasks. Such knowledge speeds up the hyperparameter tuning or feature engineering. With the use of surrogate models various aspects of the predictive task such as meta-features, landmarker models e.t.c. are used to predict the expected performance. State of the art approaches are focused on searching for the best meta-model but do not explain how these different aspects contribute to its performance. However, to build a new generation of meta-models we need a deeper …
Sloan School of Management（斯隆管理学院）；Massachusetts Institute of Technology（麻省理工学院）；Operations Research Center（运筹学研究中心）；Dep. of Electrical Engineering and Computer Science（电气工程与计算机科学系）
When predictive models are used to support complex and important decisions, the ability to explain a model's reasoning can increase trust, expose hidden biases, and reduce vulnerability to adversarial attacks. However, attempts at interpreting models are often ad hoc and application-specific, and the concept of interpretability itself is not well-defined. We propose a general optimization framework to create explanations for linear models. Our methodology decomposes a linear model into a sequence of models of increasing complexity using coordinate updates on the coefficients. Computing this de…
