We enhance the accuracy and generalization of univariate time series point prediction by an explainable ensemble on the fly. We propose an Interpretable Dynamic Ensemble Architecture (IDEA), in which interpretable base learners give predictions independently with sparse communication as a group. The model is composed of several sequentially stacked groups connected by group backcast residuals and recurrent input competition. Ensemble driven by end-to-end training both horizontally and vertically brings state-of-the-art (SOTA) performances. Forecast accuracy improves by 2.6% over the best statistical benchmark on the TOURISM dataset and 2% over the best deep learning benchmark on the M4 dataset. The architecture enjoys several advantages, being applicable to time series from various domains, explainable to users with specialized modular structure and robust to changes in task distribution.
Université Laval（拉瓦尔大学）；Element AI, a ServiceNow company（ServiceNow旗下Element AI公司）
Background: Understanding the relationship between the Omics and the phenotype is a central problem in precision medicine. The high dimensionality of metabolomics data challenges learning algorithms in terms of scalability and generalization. Most learning algorithms do not produce interpretable models -- Method: We propose an ensemble learning algorithm based on conjunctions or disjunctions of decision rules. -- Results : Applications on metabolomics data shows that it produces models that achieves high predictive performances. The interpretability of the models makes them useful for biomarke…
Diversity is an important consideration in the construction of robust neural network ensembles. A collection of well trained models will generalize better if they are diverse in the patterns they respond to and the predictions they make. Diversity is especially important for low-cost ensemble methods because members often share network structure in order to avoid training several independent models from scratch. Diversity is traditionally analyzed by measuring differences between the outputs of models. However, this gives little insight into how knowledge representations differ between ensembl…
University of Zurich（苏黎世大学）；Zurich University of Applied Sciences（苏黎世应用科学大学）；ETH Zurich（苏黎世联邦理工学院）
Ensembles improve prediction performance and allow uncertainty quantification by aggregating predictions from multiple models. In deep ensembling, the individual models are usually black box neural networks, or recently, partially interpretable semi-structured deep transformation models. However, interpretability of the ensemble members is generally lost upon aggregation. This is a crucial drawback of deep ensembles in high-stake decision fields, in which interpretable models are desired. We propose a novel transformation ensemble which aggregates probabilistic predictions with the guarantee t…
We propose to use boosted regression trees as a way to compute human-interpretable solutions to reinforcement learning problems. Boosting combines several regression trees to improve their accuracy without significantly reducing their inherent interpretability. Prior work has focused independently on reinforcement learning and on interpretable machine learning, but there has been little progress in interpretable reinforcement learning. Our experimental results show that boosted regression trees compute solutions that are both interpretable and match the quality of leading reinforcement learnin…
We present a novel method - LIBRE - to learn an interpretable classifier, which materializes as a set of Boolean rules. LIBRE uses an ensemble of bottom-up weak learners operating on a random subset of features, which allows for the learning of rules that generalize well on unseen data even in imbalanced settings. Weak learners are combined with a simple union so that the final ensemble is also interpretable. Experimental results indicate that LIBRE efficiently strikes the right balance between prediction accuracy, which is competitive with black box methods, and interpretability, which is oft…
Learning Accurate, Efficient, and Interpretable MLPs on Multiplex Graphs via Node-wise Multi-View Ensemble Distillation
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；Laboratory for Big Data and Decision, National University of Defense Technology（国防科技大学大数据与决策实验室）
Ensembles of decision trees perform well on many problems, but are not interpretable. In contrast to existing approaches in interpretability that focus on explaining relationships between features and predictions, we propose an alternative approach to interpret tree ensemble classifiers by surfacing representative points for each class -- prototypes. We introduce a new distance for Gradient Boosted Tree models, and propose new, adaptive prototype selection methods with theoretical guarantees, with the flexibility to choose a different number of prototypes in each class. We demonstrate our meth…
Ghent University - imec, IDLab（根特大学 - imec, IDLab）
Models obtained by decision tree induction techniques excel in being interpretable.However, they can be prone to overfitting, which results in a low predictive performance. Ensemble techniques are able to achieve a higher accuracy. However, this comes at a cost of losing interpretability of the resulting model. This makes ensemble techniques impractical in applications where decision support, instead of decision making, is crucial. To bridge this gap, we present the GENESIM algorithm that transforms an ensemble of decision trees to a single decision tree with an enhanced predictive performance…
Rule ensembles are designed to provide a useful trade-off between predictive accuracy and model interpretability. However, the myopic and random search components of current rule ensemble methods can compromise this goal: they often need more rules than necessary to reach a certain accuracy level or can even outright fail to accurately model a distribution that can actually be described well with a few rules. Here, we present a novel approach aiming to fit rule ensembles of maximal predictive power for a given ensemble size (and thus model comprehensibility). In particular, we present an effic…
A new ensemble framework for interpretable model called Linear Iterative Feature Embedding (LIFE) has been developed to achieve high prediction accuracy, easy interpretation and efficient computation simultaneously. The LIFE algorithm is able to fit a wide single-hidden-layer neural network (NN) accurately with three steps: defining the subsets of a dataset by the linear projections of neural nodes, creating the features from multiple narrow single-hidden-layer NNs trained on the different subsets of the data, combining the features with a linear model. The theoretical rationale behind LIFE is…
Beijing Key Laboratory of Security and Privacy in Intelligent Transportation, Beijing Jiaotong University（北京交通大学智能交通信息安全与隐私保护北京市重点实验室）
Deep learning (DL) models for image-based malware detection have exhibited their capability in producing high prediction accuracy. But model interpretability is posing challenges to their widespread application in security and safety-critical application domains. This paper aims for designing an Interpretable Ensemble learning approach for image-based Malware Detection (IEMD). We first propose a Selective Deep Ensemble Learning-based (SDEL) detector and then design an Ensemble Deep Taylor Decomposition (EDTD) approach, which can give the pixel-level explanation to SDEL detector outputs. Furthe…
Peter the Great St.Petersburg Polytechnic University（彼得大帝圣彼得堡理工大学）
A new extremely simple ensemble-based model with the uniformly generated axis-parallel hyper-rectangles as base models (HRBM) is proposed. Two types of HRBMs are studied: closed rectangles and corners. The main idea behind HRBM is to consider and count training examples inside and outside each rectangle. It is proposed to incorporate HRBMs into the gradient boosting machine (GBM). Despite simplicity of HRBMs, it turns out that these simple base models allow us to construct effective ensemble-based models and avoid overfitting. A simple method for calculating optimal regularization parameters o…
Interpretable Representation Learning for Additive Rule Ensembles
Department of Data Science and Artificial Intelligence, Monash University（墨尔本大学数据科学与人工智能系）；Department of Information Systems, University of Haifa（海法大学信息系统系）
Interpretable Ensemble Learning for Materials Property Prediction with Classical Interatomic Potentials: Carbon as an Example
School for Engineering of Matter, Transport and Energy, Arizona State University（亚利桑那州立大学物质、运输与能源工程学院）；Material Measurement Laboratory, National Institute of Standards and Technology（美国国家标准与技术研究院材料测量实验室）
REALITrees: Rashomon Ensemble Active Learning for Interpretable Trees
Department of Statistics, University of Washington（统计学系，华盛顿大学）；Department of Computer Science, Duke University（计算机科学系，杜克大学）；Department of Sociology, University of Washington（社会学系，华盛顿大学）
Peter the Great St.Petersburg Polytechnic University（圣彼得堡彼得大帝理工大学）
A method for the local and global interpretation of a black-box model on the basis of the well-known generalized additive models is proposed. It can be viewed as an extension or a modification of the algorithm using the neural additive model. The method is based on using an ensemble of gradient boosting machines (GBMs) such that each GBM is learned on a single feature and produces a shape function of the feature. The ensemble is composed as a weighted sum of separate GBMs resulting a weighted sum of shape functions which form the generalized additive model. GBMs are built in parallel using ran…
An Interpretable Ensemble Framework for Multi-Omics Dementia Biomarker Discovery Under HDLSS Conditions
Department of Mathematics and Physics, Mathematics Major, Gangneung-Wonju National University（数学与物理系，数学专业，江陵-Wonju国立大学）；Department of Data Science, Gangneung-Wonju National University（数据科学系，江陵-Wonju国立大学）
Interpretable Physics-Informed Load Forecasting for U.S. Grid Resilience: SHAP-Guided Ensemble Validation in Hybrid Deep Learning Under Extreme Weather
1Dept. of Computer Science, Midwestern State University, Dallas, TX, USA 2O\&M Analytics, AES Clean Energy, The AES Corporation, Louisville, CO, USA 3Dept. of Information；Communication Engineering, Pabna University of Science
StackLiverNet: A Novel Stacked Ensemble Model for Accurate and Interpretable Liver Disease Detection
East West University（东西大学）；Bangladesh Agricultural University（孟加拉国农业大学）；Daffodil International University（达福迪尔国际大学）；University of Dhaka（达卡大学）；University of Barishal（巴里萨尔大学）；Cumilla Victoria Government College（库米拉维多利亚政府学院）；Bangladesh University of Business and Technology（孟加拉国商业与技术大学）
Random Rule Forest (RRF): Interpretable and Manageable Ensembles of LLM-Generated Questions for Predicting Success from Unstructured Data
University of Oxford, United Kingdom Aarhus University, Aarhus, Denmark Centre de Recerca Matem\`atica, Barcelona, Spain Oliver Wyman, New York, United States Vela Research, San Francisco, United States
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Hospital of Sickkids（病童医院）；Microsoft Research（微软研究院）
Deployment of machine learning models in real high-risk settings (e.g. healthcare) often depends not only on the model's accuracy but also on its fairness, robustness, and interpretability. Generalized Additive Models (GAMs) are a class of interpretable models with a long history of use in these high-risk domains, but they lack desirable features of deep learning such as differentiability and scalability. In this work, we propose a neural GAM (NODE-GAM) and neural GA$^2$M (NODE-GA$^2$M) that scale well and perform better than other GAMs on large datasets, while remaining interpretable compared…
Climate-resilient electric vehicle charging infrastructure for sustainable cities: An interpretable causal-ensemble framework for preventive maintenance and low-carbon mobility
Foshan University（佛山科学技术学院）；Jilin University（吉林大学）；National University of Singapore（新加坡国立大学）
Pontifical Catholic University of Rio de Janeiro (PUC-Rio)（里约热内卢天主教大学）；Technical University of Munich（慕尼黑工业大学）
The use of machine learning algorithms in finance, medicine, and criminal justice can deeply impact human lives. As a consequence, research into interpretable machine learning has rapidly grown in an attempt to better control and fix possible sources of mistakes and biases. Tree ensembles offer a good prediction quality in various domains, but the concurrent use of multiple trees reduces the interpretability of the ensemble. Against this background, we study born-again tree ensembles, i.e., the process of constructing a single decision tree of minimum size that reproduces the exact same behavi…
CoreML Group Apple Inc.（苹果公司CoreML团队）；Department of Mathematics Florida State University（佛罗里达州立大学数学系）；Department of Statistics Florida State University（佛罗里达州立大学统计系）
Tree ensembles are flexible predictive models that can capture relevant variables and to some extent their interactions in a compact and interpretable manner. Most algorithms for obtaining tree ensembles are based on versions of boosting or Random Forest. Previous work showed that boosting algorithms exhibit a cyclic behavior of selecting the same tree again and again due to the way the loss is optimized. At the same time, Random Forest is not based on loss optimization and obtains a more complex and less interpretable model. In this paper we present a novel method for obtaining compact tree e…
University of Twente（特文特大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
Prototype-based methods use interpretable representations to address the black-box nature of deep learning models, in contrast to post-hoc explanation methods that only approximate such models. We propose the Neural Prototype Tree (ProtoTree), an intrinsically interpretable deep learning method for fine-grained image recognition. ProtoTree combines prototype learning with decision trees, and thus results in a globally interpretable model by design. Additionally, ProtoTree can locally explain a single prediction by outlining a decision path through the tree. Each node in our binary tree contain…
Department of Medical Data Science（医学数据科学系）；Wakayama Medical University（福冈医疗大学）；Department of Biomedical Sciences and Informatics（生物医学科学与信息学系）；Doshisha University（多摩大学）；Department of Biostatistics（生物统计学系）
Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning
Cooperative Innovation Center, Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所协同创新中心）；University of Chinese Academy of Sciences（中国科学院大学）；JD Technology Group（京东科技集团）；University of Technology Sydney（悉尼科技大学）
Optimizing Stroke Risk Prediction: A Machine Learning Pipeline Combining ROS-Balanced Ensembles and XAI
1 Department of Robotics, Robo Tech Valley, Dhaka, Bangladesh 2 Department of Computer Science；Engineering, Jagannath University, Dhaka, Bangladesh 3 Department of Internet of Things \& Robotics Engineering, University of Frontier Technology, Bangladesh
Interpretable epistemic uncertainty decomposition in sequential generative models via polynomial chaos surrogates
Mathematical Institute, University of Oxford, United Kingdom（牛津大学数学研究所，英国）；IBM Research, Thomas J. Watson Research Center, USA（IBM研究院，托马斯·J·沃森研究中心，美国）；IBM Research Europe, Daresbury, United Kingdom（IBM欧洲研究院，达尔斯伯里，英国）；MIT–IBM Watson AI Lab, Massachusetts, USA（麻省理工–IBM沃森人工智能实验室，马萨诸塞州，美国）；Université de Montréal, Mila–Quebec AI Institute, Montreal, Quebec, Canada（蒙特利尔大学，魁北克人工智能研究所，蒙特利尔，魁北克，加拿大）
Norwegian University of Science and Technology (NTNU)（挪威科技大学）
Prediction intervals are a machine- and human-interpretable way to represent predictive uncertainty in a regression analysis. In this paper, we present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results and stabilizes the training process of the neural networks. The ensembled prediction intervals are aggregated as a split normal mixture accounting…
Electronic Health Records (EHRs) have been heavily used to predict various downstream clinical tasks such as readmission or mortality. One of the modalities in EHRs, clinical notes, has not been fully explored for these tasks due to its unstructured and inexplicable nature. Although recent advances in deep learning (DL) enables models to extract interpretable features from unstructured data, they often require a large amount of training data. However, many tasks in medical domains inherently consist of small sample data with lengthy documents; for a kidney transplant as an example, data from o…
Ensemble learning is a mainstay in modern data science practice. Conventional ensemble algorithms assigns to base models a set of deterministic, constant model weights that (1) do not fully account for variations in base model accuracy across subgroups, nor (2) provide uncertainty estimates for the ensemble prediction, which could result in mis-calibrated (i.e. precise but biased) predictions that could in turn negatively impact the algorithm performance in real-word applications. In this work, we present an adaptive, probabilistic approach to ensemble learning using dependent tail-free proces…
