Complex black-box predictive models may have high performance, but lack of interpretability causes problems like lack of trust, lack of stability, sensitivity to concept drift. On the other hand, achieving satisfactory accuracy of interpretable models require more time-consuming work related to feature engineering. Can we train interpretable and accurate models, without timeless feature engineering? We propose a method that uses elastic black-boxes as surrogate models to create a simpler, less opaque, yet still accurate and interpretable glass-box models. New models are created on newly engineered features extracted with the help of a surrogate model. We supply the analysis by a large-scale benchmark on several tabular data sets from the OpenML database. There are two results 1) extracting information from complex models may improve the performance of linear models, 2) questioning a common myth that complex machine learning models outperform linear models.
University of Waterloo（滑铁卢大学）；Vector Institute；University of Toronto（多伦多大学）
To date, there has been no formal study of the statistical cost of interpretability in machine learning. As such, the discourse around potential trade-offs is often informal and misconceptions abound. In this work, we aim to initiate a formal study of these trade-offs. A seemingly insurmountable roadblock is the lack of any agreed upon definition of interpretability. Instead, we propose a shift in perspective. Rather than attempt to define interpretability, we propose to model the \emph{act} of \emph{enforcing} interpretability. As a starting point, we focus on the setting of empirical risk mi…
We analyze the trade-off between model complexity and accuracy for random forests by breaking the trees up into individual classification rules and selecting a subset of them. We show experimentally that already a few rules are sufficient to achieve an acceptable accuracy close to that of the original model. Moreover, our results indicate that in many cases, this can lead to simpler models that clearly outperform the original ones.
Graph clustering groups entities -- the vertices of a graph -- based on their similarity, typically using a complex distance function over a large number of features. Successful integration of clustering approaches in automated decision-support systems hinges on the interpretability of the resulting clusters. This paper addresses the problem of generating interpretable clusters, given features of interest that signify interpretability to an end-user, by optimizing interpretability in addition to common clustering objectives. We propose a $β$-interpretable clustering algorithm that ensures that…
Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, which lack guarantees about their explanation quality. We propose an alternative to these approaches by directly regularizing a black-box model for interpretability at training time. Our approach explicitly connects three key aspects of interpretable machine learning: (i) the model's innate explainability, (ii) the explanation system used at test time, and (iii) the metrics that measure expl…
Massachusetts Institute of Technology（麻省理工学院）；IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
As a contribution to interpretable machine learning research, we develop a novel optimization framework for learning accurate and sparse two-level Boolean rules. We consider rules in both conjunctive normal form (AND-of-ORs) and disjunctive normal form (OR-of-ANDs). A principled objective function is proposed to trade classification accuracy and interpretability, where we use Hamming loss to characterize accuracy and sparsity to characterize interpretability. We propose efficient procedures to optimize these objectives based on linear programming (LP) relaxation, block coordinate descent, and …
American University（美利坚大学）；University of Maryland（马里兰大学）；Energetics Technology Center（含能材料技术中心）
As the amount and variety of energetics research increases, machine aware topic identification is necessary to streamline future research pipelines. The makeup of an automatic topic identification process consists of creating document representations and performing classification. However, the implementation of these processes on energetics research imposes new challenges. Energetics datasets contain many scientific terms that are necessary to understand the context of a document but may require more complex document representations. Secondly, the predictions from classification must be unders…
Sloan School of Management（斯隆管理学院）；Massachusetts Institute of Technology（麻省理工学院）；Operations Research Center（运筹学研究中心）；Dep. of Electrical Engineering and Computer Science（电气工程与计算机科学系）
When quantitative models are used to support decision-making on complex and important topics, understanding a model's ``reasoning'' can increase trust in its predictions, expose hidden biases, or reduce vulnerability to adversarial attacks. However, the concept of interpretability remains loosely defined and application-specific. In this paper, we introduce a mathematical framework in which machine learning models are constructed in a sequence of interpretable steps. We show that for a variety of models, a natural choice of interpretable steps recovers standard interpretability proxies (e.g., …
Carnegie Mellon University（卡内基梅隆大学）；Petuum Inc（Petuum公司）；Determined AI
Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, whose explanation quality can be unpredictable. Our method, ExpO, is a hybridization of these approaches that regularizes a model for explanation quality at training time. Importantly, these regularizers are differentiable, model agnostic, and require no domain knowledge to define. We demonstrate that post-hoc explanations for ExpO-regularized models have better explanation quality, as measu…
Tackling the Accuracy-Interpretability Trade-off in a Hierarchy of Machine Learning Models for the Prediction of Extreme Heatwaves
ENS de Lyon（里昂高等师范学院）；CNRS（法国国家科学研究中心）；LMD/IPSL（巴黎气候与环境科学研究所）；ENS（高等师范学院）；Université PSL（巴黎文理大学）；École Polytechnique（巴黎综合理工学院）；Institut Polytechnique de Paris（巴黎理工学院）；Sorbonne Université（索邦大学）；RTE France（法国输电网络公司）
Centrum Wiskunde & Informatica（荷兰国家数学与计算机科学研究所）；University of Trieste（的里雅斯特大学）；IMT School for Advanced Studies Lucca（卢卡IMT高等研究院）
Many risk-sensitive applications require Machine Learning (ML) models to be interpretable. Attempts to obtain interpretable models typically rely on tuning, by trial-and-error, hyper-parameters of model complexity that are only loosely related to interpretability. We show that it is instead possible to take a meta-learning approach: an ML model of non-trivial Proxies of Human Interpretability (PHIs) can be learned from human feedback, then this model can be incorporated within an ML training process to directly optimize for interpretability. We show this for evolutionary symbolic regression. W…
Challenging the Performance-Interpretability Trade-off: An Evaluation of Interpretable Machine Learning Models
University of Regensburg（雷根斯堡大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Technische Hochschule Nürnberg Georg Simon Ohm（纽伦堡应用技术大学乔治·西蒙·欧姆分校）；Leipzig University（莱比锡大学）
Insight Centre for Data Analytics, University College Dublin（都柏林大学学院洞察数据分析中心）；Accenture Labs（埃森哲实验室）；Inria（法国国家信息与自动化研究所）
Sequence classification is the supervised learning task of building models that predict class labels of unseen sequences of symbols. Although accuracy is paramount, in certain scenarios interpretability is a must. Unfortunately, such trade-off is often hard to achieve since we lack human-independent interpretability metrics. We introduce a novel sequence learning algorithm, that combines (i) linear classifiers - which are known to strike a good balance between predictive power and interpretability, and (ii) background knowledge embeddings. We extend the classic subsequence feature space with g…
Sloan School of Management（斯隆管理学院）；Massachusetts Institute of Technology（麻省理工学院）；Operations Research Center（运筹学研究中心）；Dep. of Electrical Engineering and Computer Science（电气工程与计算机科学系）
When predictive models are used to support complex and important decisions, the ability to explain a model's reasoning can increase trust, expose hidden biases, and reduce vulnerability to adversarial attacks. However, attempts at interpreting models are often ad hoc and application-specific, and the concept of interpretability itself is not well-defined. We propose a general optimization framework to create explanations for linear models. Our methodology decomposes a linear model into a sequence of models of increasing complexity using coordinate updates on the coefficients. Computing this de…
Interpretable deep learning is a fundamental building block towards safer AI, especially when the deployment possibilities of deep learning-based computer-aided medical diagnostic systems are so eminent. However, without a computational formulation of black-box interpretation, general interpretability research rely heavily on subjective bias. Clear decision structure of the medical diagnostics lets us approximate the decision process of a radiologist as a model - removed from subjective bias. We define the process of interpretation as a finite communication between a known model and a black-bo…
As Convolutional Neural Networks embed themselves into our everyday lives, the need for them to be interpretable increases. However, there is often a trade-off between methods that are efficient to compute but produce an explanation that is difficult to interpret, and those that are slow to compute but provide a more interpretable result. This is particularly challenging in problem spaces that require a large input volume, especially video which combines both spatial and temporal dimensions. In this work we introduce the idea of scoring superpixels through the use of gradient based pixel scori…
Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning
Southern University of Science and Technology（南方科技大学）；Beijing Normal-Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；Nanyang Technological University（南洋理工大学）；Lingnan University（岭南大学）
Helsinki Institute for Information Technology HIIT（赫尔辛基信息技术研究所）；Aalto University（阿尔托大学）；Curious AI（Curious AI 公司）；University of Manchester（曼彻斯特大学）
A salient approach to interpretable machine learning is to restrict modeling to simple models. In the Bayesian framework, this can be pursued by restricting the model structure and prior to favor interpretable models. Fundamentally, however, interpretability is about users' preferences, not the data generation mechanism; it is more natural to formulate interpretability as a utility function. In this work, we propose an interpretability utility, which explicates the trade-off between explanation fidelity and interpretability in the Bayesian framework. The method consists of two steps. First, a …
Chalmers University of Technology（查尔姆斯理工大学）；University of Trieste（的里雅斯特大学）；IMT Lucca（IMT卢卡研究院）
High-stakes applications require AI-generated models to be interpretable. Current algorithms for the synthesis of potentially interpretable models rely on objectives or regularization terms that represent interpretability only coarsely (e.g., model size) and are not designed for a specific user. Yet, interpretability is intrinsically subjective. In this paper, we propose an approach for the synthesis of models that are tailored to the user by enabling the user to steer the model synthesis process according to her or his preferences. We use a bi-objective evolutionary algorithm to synthesize mo…
Data-driven models are central to scientific discovery. In efforts to achieve state-of-the-art model accuracy, researchers are employing increasingly complex machine learning algorithms that often outperform simple regressions in interpolative settings (e.g. random k-fold cross-validation) but suffer from poor extrapolation performance, portability, and human interpretability, which limits their potential for facilitating novel scientific insight. Here we examine the trade-off between model performance and interpretability across a broad range of science and engineering problems with an emphas…
Real-world applications of reinforcement learning for recommendation and experimentation faces a practical challenge: the relative reward of different bandit arms can evolve over the lifetime of the learning agent. To deal with these non-stationary cases, the agent must forget some historical knowledge, as it may no longer be relevant to minimise regret. We present a solution to handling non-stationarity that is suitable for deployment at scale, to provide business operators with automated adaptive optimisation. Our solution aims to provide interpretable learning that can be trusted by humans,…
Contextualized Policy Recovery: Modeling and Interpreting Medical Decisions with Adaptive Imitation Learning
Carnegie Mellon University（卡内基梅隆大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Broad Institute of MIT and Harvard（麻省理工学院与哈佛大学博德研究所）；MIT（麻省理工学院）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Petuum, Inc.（Petuum公司）
IBM Research（IBM研究院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；Michigan State University（密歇根州立大学）；Cornell University（康奈尔大学）
Drug similarity has been studied to support downstream clinical tasks such as inferring novel properties of drugs (e.g. side effects, indications, interactions) from known properties. The growing availability of new types of drug features brings the opportunity of learning a more comprehensive and accurate drug similarity that represents the full spectrum of underlying drug relations. However, it is challenging to integrate these heterogeneous, noisy, nonlinear-related information to learn accurate similarity measures especially when labels are scarce. Moreover, there is a trade-off between ac…
Towards Accurate and Interpretable Time-series Forecasting: A Polynomial Learning Approach
Center for Intelligent Decision-Making（智能决策中心）；Machine Learning, School of Management, Xi'an Jiaotong University, Xi'an, China（机器学习，管理学院，西安交通大学，中国）；The 39th Research Institute of China Electronics Technology Group Corporation, Xi'an, China（中国电子科技集团第三十九研究所，中国）；Shenzhen Research Institute of Big Data, ShenZhen, China（深圳大数据研究院，中国）
Tippie College of Business（蒂皮商学院）；University of Iowa（爱荷华大学）
Interpretable machine learning has become a strong competitor for traditional black-box models. However, the possible loss of the predictive performance for gaining interpretability is often inevitable, putting practitioners in a dilemma of choosing between high accuracy (black-box models) and interpretability (interpretable models). In this work, we propose a novel framework for building a Hybrid Predictive Model (HPM) that integrates an interpretable model with any black-box model to combine their strengths. The interpretable model substitutes the black-box model on a subset of data where th…
We present an approach to improve the accuracy-interpretability trade-off of Machine Learning (ML) Decision Trees (DTs). In particular, we apply Maximum Satisfiability technology to compute Minimum Pure DTs (MPDTs). We improve the runtime of previous approaches and, show that these MPDTs can outperform the accuracy of DTs generated with the ML framework sklearn.
Aalto University（阿尔托大学）；KTH Royal Institute of Technology（KTH皇家理工学院）
Multi-label classification is becoming increasingly ubiquitous, but not much attention has been paid to interpretability. In this paper, we develop a multi-label classifier that can be represented as a concise set of simple "if-then" rules, and thus, it offers better interpretability compared to black-box models. Notably, our method is able to find a small set of relevant patterns that lead to accurate multi-label classification, while existing rule-based classifiers are myopic and wasteful in searching rules,requiring a large number of rules to achieve high accuracy. In particular, we formula…
Beijing Key Laboratory of Security and Privacy in Intelligent Transportation, Beijing Jiaotong University（北京交通大学智能交通信息安全与隐私保护北京市重点实验室）
Deep learning (DL) models for image-based malware detection have exhibited their capability in producing high prediction accuracy. But model interpretability is posing challenges to their widespread application in security and safety-critical application domains. This paper aims for designing an Interpretable Ensemble learning approach for image-based Malware Detection (IEMD). We first propose a Selective Deep Ensemble Learning-based (SDEL) detector and then design an Ensemble Deep Taylor Decomposition (EDTD) approach, which can give the pixel-level explanation to SDEL detector outputs. Furthe…
PISA: An AI Pipeline for Interpretable-by-design Survival Analysis Providing Multiple Complexity-Accuracy Trade-off Models
a Leiden University Medical Center, Leiden, The Netherlands；b Amsterdam University Medical Centers location Vrije Universiteit Amsterdam, Amsterdam, The Netherlands；c Cancer Center Amsterdam, Cancer Treatment；Quality of Life, Amsterdam, The Netherlands；d Centrum Wiskunde \& Informatica, Amsterdam, The Netherlands
Georgia Institute of Technology（佐治亚理工学院）；Sutter Health（萨特健康）
Accuracy and interpretability are two dominant features of successful predictive models. Typically, a choice must be made in favor of complex black box models such as recurrent neural networks (RNN) for accuracy versus less accurate but more interpretable traditional models such as logistic regression. This tradeoff poses challenges in medicine where both accuracy and interpretability are important. We addressed this challenge by developing the REverse Time AttentIoN model (RETAIN) for application to Electronic Health Records (EHR) data. RETAIN achieves high accuracy while remaining clinically…
