Today, it is more important than ever before for users to have trust in the models they use. As Machine Learning models fall under increased regulatory scrutiny and begin to see more applications in high-stakes situations, it becomes critical to explain our models. Piecewise Linear Neural Networks (PLNN) with the ReLU activation function have quickly become extremely popular models due to many appealing properties; however, they still present many challenges in the areas of robustness and interpretation. To this end, we introduce novel methodology toward simplification and increased interpretability of Piecewise Linear Neural Networks for classification tasks. Our methods include the use of a trained, deep network to produce a well-performing, single-hidden-layer network without further stochastic training, in addition to an algorithm to reduce flat networks to a smaller, more interpretable size with minimal loss in performance. On these methods, we conduct preliminary studies of model performance, as well as a case study on Wells Fargo's Home Lending dataset, together with visual model interpretation.
Xi’an Jiaotong University（西安交通大学）；City University of Hong Kong（香港城市大学）；Chinese Academy of Sciences（中国科学院）；The University of Arizona（亚利桑那大学）
Most existing interpretable methods explain a black-box model in a post-hoc manner, which uses simpler models or data analysis techniques to interpret the predictions after the model is learned. However, they (a) may derive contradictory explanations on the same predictions given different methods and data samples, and (b) focus on using simpler models to provide higher descriptive accuracy at the sacrifice of prediction accuracy. To address these issues, we propose a hybrid interpretable model that combines a piecewise linear component and a nonlinear component. The first component describes …
Deep learning models have achieved state-of-the-art performance in many classification tasks. However, most of them cannot provide an interpretation for their classification results. Machine learning models that are interpretable are usually linear or piecewise linear and yield inferior performance. Non-linear models achieve much better classification performance, but it is hard to interpret their classification results. This may have been changed by an interpretable feedforward neural network (IFFNN) proposed that achieves both high classification performance and interpretability for malware …
PHEATPRUNER: Interpretable Data-centric Feature Selection for Multivariate Time Series Classification through Persistent Homology
Osnabrück University, Joint Lab Artificial Intelligence & Data Science（奥斯纳布鲁克大学，人工智能与数据科学联合实验室）；Osnabrück University, Institute of Mathematics（奥斯纳布鲁克大学，数学学院）；Osnabrück University, Semantic Information Systems Group（奥斯纳布鲁克大学，语义信息系统组）；German Research Center for Artificial Intelligence (DFKI), Research Department Plan-Based Robot Control（德国人工智能研究中心（DFKI），计划型机器人控制研究部）；Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB), Department of Sensors and Modelling（莱比锡农业工程与生物经济研究所（ATB），传感器与建模部门）
KJ Somaiya College of Engineering（KJ Somaiya工程学院）；National Institute of Technology Karnataka, Surathkal（卡纳塔克国立理工学院苏拉特卡尔分校）
Relu Fully Connected Networks are ubiquitous but uninterpretable because they fit piecewise linear functions emerging from multi-layered structures and complex interactions of model weights. This paper takes a novel approach to piecewise fits by using set operations on individual pieces(parts). This is done by approximating canonical normal forms and using the resultant as a model. This gives special advantages like (a)strong correspondence of parameters to pieces of the fit function(High Interpretability); (b)ability to fit any combination of continuous functions as pieces of the piecewise fu…
Interpretable Physics-Informed Load Forecasting for U.S. Grid Resilience: SHAP-Guided Ensemble Validation in Hybrid Deep Learning Under Extreme Weather
1Dept. of Computer Science, Midwestern State University, Dallas, TX, USA 2O\&M Analytics, AES Clean Energy, The AES Corporation, Louisville, CO, USA 3Dept. of Information；Communication Engineering, Pabna University of Science
CLARITree: Cholesky and Lookahead Accelerations for Regression with Interpretable Piecewise Linear Trees
Department of Computer Science, Duke University, Durham, USA（杜克大学计算机科学系，美国达勒姆）；Department of Computer Science, University of British Columbia, Vancouver, Canada（不列颠哥伦比亚大学计算机科学系，加拿大温哥华）
Cornell University（康奈尔大学）；Federal Institute of Ceara（塞阿拉联邦学院）
Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear c…
The goal of model distillation is to faithfully transfer teacher model knowledge to a model which is faster, more generalizable, more interpretable, or possesses other desirable characteristics. Human-readability is an important and desirable standard for machine-learned model interpretability. Readable models are transparent and can be reviewed, manipulated, and deployed like traditional source code. As a result, such models can be improved outside the context of machine learning and manually edited if desired. Given that directly training such models is difficult, we propose to train interpr…
Interpretable and explainable machine learning has seen a recent surge of interest. We focus on safety as a key motivation behind the surge and make the relationship between interpretability and safety more quantitative. Toward assessing safety, we introduce the concept of maximum deviation via an optimization problem to find the largest deviation of a supervised learning model from a reference model regarded as safe. We then show how interpretability facilitates this safety assessment. For models including decision trees, generalized linear and additive models, the maximum deviation can be co…
Barycentric Neural Networks and Length-Weighted Persistent Entropy Loss: A Green Geometric and Topological Framework for Function Approximation
Department of Applied Mathematics I, University of Seville（应用数学I系，塞维利亚大学）；Department of Computer Science and Artificial Intelligence, University of Seville（计算机科学与人工智能系，塞维利亚大学）
VeriFlow: Modeling Distributions for Neural Network Verification
Independent Researcher（独立研究者）；TU Dortmund University（图卢兹-多特蒙德大学）；Center for Trustworthy Data Science and Security（可信数据科学与安全中心）；University Alliance Ruhr（鲁尔大学联盟）
Aston University（阿斯顿大学）；University of Sussex（萨塞克斯大学）；University of Birmingham（伯明翰大学）；Massachusetts Institute of Technology（麻省理工学院）
Ubiquitous linear Gaussian exploratory tools such as principle component analysis (PCA) and factor analysis (FA) remain widely used as tools for: exploratory analysis, pre-processing, data visualization and related tasks. However, due to their rigid assumptions including crowding of high dimensional data, they have been replaced in many settings by more flexible and still interpretable latent feature models. The Feature allocation is usually modelled using discrete latent variables assumed to follow either parametric Beta-Bernoulli distribution or Bayesian nonparametric prior. In this work we …
Faculty of Physics and Astronomy, Heidelberg University（海德堡大学物理与天文学学院）；Institute of Neuroinformatics, University of Zurich and ETH Zurich（苏黎世大学与苏黎世联邦理工学院神经信息学研究所）
A major tenet in theoretical neuroscience is that cognitive and behavioral processes are ultimately implemented in terms of the neural system dynamics. Accordingly, a major aim for the analysis of neurophysiological measurements should lie in the identification of the computational dynamics underlying task processing. Here we advance a state space model (SSM) based on generative piecewise-linear recurrent neural networks (PLRNN) to assess dynamics from neuroimaging data. In contrast to many other nonlinear time series models which have been proposed for reconstructing latent dynamics, our mode…
Networks have become indispensable and ubiquitous structures in many fields to model the interactions among different entities, such as friendship in social networks or protein interactions in biological graphs. A major challenge is to understand the structure and dynamics of these systems. Although networks evolve through time, most existing graph representation learning methods target only static networks. Whereas approaches have been developed for the modeling of dynamic networks, there is a lack of efficient continuous time dynamic graph representation learning methods that can provide acc…
Mederrata Research and Sound Prediction（Mederrata研究与声音预测公司）；NIH NIDDK（美国国立卫生研究院国家糖尿病、消化与肾脏疾病研究所）；Mederrata Research（Mederrata研究公司）；Blue Health Intelligence（蓝色健康智能公司）；Walter Reed Army Institute of Research（沃尔特·里德陆军研究所）；Sound Prediction and Mederrata Research（声音预测与Mederrata研究公司）
We developed an inherently interpretable multilevel Bayesian framework for representing variation in regression coefficients that mimics the piecewise linearity of ReLU-activated deep neural networks. We used the framework to formulate a survival model for using medical claims to predict hospital readmission and death that focuses on discharge placement, adjusting for confounding in estimating causal local average treatment effects. We trained the model on a 5% sample of Medicare beneficiaries from 2008 and 2011, based on their 2009--2011 inpatient episodes, and then tested the model on 2012 e…
Locally Linear Continual Learning for Time Series based on VC-Theoretical Generalization Bounds
Department of Electronic Engineering, Universidade Federal de Minas Gerais（米纳斯格拉斯联邦大学电子工程系）；Graduate Program in Electrical Engineering, Universidade Federal de Minas Gerais（米纳斯格拉斯联邦大学电气工程研究生项目）
Embedding interpretable $\ell_1$-regression into neural networks for uncovering temporal structure in cell imaging
Institute of Medical Biometry and Statistics (IMBI), Faculty of Medicine and Medical Center, University of Freiburg, Germany（医学生物度量与统计研究所（IMBI）、医学学院与医疗中心、弗赖堡大学）；Institute of Physiology I, Faculty of Medicine, University of Freiburg, Germany（医学生理学研究所I、医学学院、弗赖堡大学）；Department of Computer Science, University of Freiburg, Germany（计算机科学系、弗赖堡大学）；Freiburg Center for Data Analysis, Modeling and AI (FDMAI), University of Freiburg, Germany（弗赖堡数据分析、建模与人工智能中心（FDMAI）、弗赖堡大学）；Centre for Integrative Biological Signalling Studies (CIBSS), University of Freiburg, Germany（整合生物信号研究中心（CIBSS）、弗赖堡大学）
Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction
Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University（海德堡大学跨学科科学计算中心）；Faculty of Physics and Astronomy, Heidelberg University（海德堡大学物理与天文学院）
Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks
Physics, Chalmers University of Technology；University of Gothenburg；Ecole Normale Superieure, PSL \& CNRS；Engineering, Chalmers University of Technology；School of Computer Science；Applied Mathematics, University of the Witwatersrand
École polytechnique fédérale de Lausanne（洛桑联邦理工学院）
Regression is one of the core problems tackled in supervised learning. Rectified linear unit (ReLU) neural networks generate continuous and piecewise-linear (CPWL) mappings and are the state-of-the-art approach for solving regression problems. In this paper, we propose an alternative method that leverages the expressivity of CPWL functions. In contrast to deep neural networks, our CPWL parameterization guarantees stability and is interpretable. Our approach relies on the partitioning of the domain of the CPWL function by a Delaunay triangulation. The function values at the vertices of the tria…
