TIME: A Transparent, Interpretable, Model-Adaptive and Explainable Neural Network for Dynamic Physical Processes
Oden Institute for Computational Engineering and Sciences（奥登计算工程与科学研究所）；Department of Computer Science（计算机科学系）；School of Information（信息学院）；University of Texas, Austin（得克萨斯大学奥斯汀分校）
Gurpreet Singh, Soumyajit Gupta, Matt Lease, Clint N. Dawson
Partial Differential Equations are infinite dimensional encoded representations of physical processes. However, imbibing multiple observation data towards a coupled representation presents significant challenges. We present a fully convolutional architecture that captures the invariant structure of the domain to reconstruct the observable system. The proposed architecture is significantly low-weight compared to other networks for such problems. Our intent is to learn coupled dynamic processes interpreted as deviations from true kernels representing isolated processes for model-adaptivity. Experimental analysis shows that our architecture is robust and transparent in capturing process kernels and system anomalies. We also show that high weights representation is not only redundant but also impacts network interpretability. Our design is guided by domain knowledge, with isolated process representations serving as ground truths for verification. These allow us to identify redundant kernels and their manifestations in activation maps to guide better designs that are both interpretable and explainable unlike traditional deep-nets.
Latent factor collaborative filtering (CF) has been a widely used technique for recommender system by learning the semantic representations of users and items. Recently, explainable recommendation has attracted much attention from research community. However, trade-off exists between explainability and performance of the recommendation where metadata is often needed to alleviate the dilemma. We present a novel feature mapping approach that maps the uninterpretable general features onto the interpretable aspect features, achieving both satisfactory accuracy and explainability in the recommendat…
Machine Learning algorithms are increasingly being used in recent years due to their flexibility in model fitting and increased predictive performance. However, the complexity of the models makes them hard for the data analyst to interpret the results and explain them without additional tools. This has led to much research in developing various approaches to understand the model behavior. In this paper, we present the Explainable Neural Network (xNN), a structured neural network designed especially to learn interpretable features. Unlike fully connected neural networks, the features engineered…
TrueGradeAI: Retrieval-Augmented and Bias-Resistant AI for Transparent and Explainable Digital Assessments
Amity Center for Artificial Intelligence, Amity University（阿米蒂大学人工智能中心，阿米蒂大学）；Amity School of Engineering and Technology, Amity University（阿米蒂工程科技学院，阿米蒂大学）
BrightMind, Bright Media Inc.（BrightMind，Bright Media 公司）
Two key challenges within Reinforcement Learning involve improving (a) agent learning within environments with sparse extrinsic rewards and (b) the explainability of agent actions. We describe a curious subgoal focused agent to address both these challenges. We use a novel method for curiosity produced from a Generative Adversarial Network (GAN) based model of environment transitions that is robust to stochastic environment transitions. Additionally, we use a subgoal generating network to guide navigation. The explainability of the agent's behavior is increased by decomposing complex tasks int…
Convolutional neural networks (CNN) are known for their excellent feature extraction capabilities to enable the learning of models from data, yet are used as black boxes. An interpretation of the convolutional filtres and associated features can help to establish an understanding of CNN to distinguish various classes. In this work, we focus on the explainability of a CNN model called as cnnexplain that is used for Covid-19 and non-Covid-19 classification with a focus on the interpretability of features by the convolutional filters, and how these features contribute to classification. Specifica…
UiT The Arctic University of Norway（挪威北极大学）；Technical University of Berlin（柏林工业大学）
The need for interpretable models has fostered the development of self-explainable classifiers. Prior approaches are either based on multi-stage optimization schemes, impacting the predictive performance of the model, or produce explanations that are not transparent, trustworthy or do not capture the diversity of the data. To address these shortcomings, we propose ProtoVAE, a variational autoencoder-based framework that learns class-specific prototypes in an end-to-end manner and enforces trustworthiness and diversity by regularizing the representation space and introducing an orthonormality c…
Applied Artificial Intelligence Institute (A2I2)（应用人工智能研究所（A2I2））；Deakin University（迪肯大学）
Deep learning has become popular because of its potential to achieve high accuracy in prediction tasks. However, accuracy is not always the only goal of statistical modelling, especially for models developed as part of scientific research. Rather, many scientific models are developed to facilitate scientific discovery, by which we mean to abstract a human-understandable representation of the natural world. Unfortunately, the opacity of deep neural networks limit their role in scientific discovery, creating a new demand for models that are transparently interpretable. This article is a field gu…
Center for Public Health Genomics（公共卫生基因组学中心）；Department of Biomedical Engineering University of Virginia（弗吉尼亚大学生物医学工程系）；University of Virginia（弗吉尼亚大学）
White Blood Cell (WBC) Leukaemia is detected through image-based classification. Convolutional Neural Networks are used to learn the features needed to classify images of cells a malignant or healthy. However, this type of model requires learning a large number of parameters and is difficult to interpret and explain. Explainable AI (XAI) attempts to alleviate this issue by providing insights to how models make decisions. Therefore, we present an XAI model which uses only 24 explainable and interpretable features and is highly competitive to other approaches by outperforming them by about 4.38\…
CONFIDERAI: a novel CONFormal Interpretable-by-Design score function for Explainable and Reliable Artificial Intelligence
Institute of Electronics, Information Engineering and Telecommunications - National Research Council of Italy (CNR-IEIIT)（电子信息与电信研究所 - 意大利国家科研理事会（CNR-IEIIT））；Politecnico di Torino - Department of Control and Computer Engineering (DAUIN)（托尼诺理工学院 - 控制与计算机工程系（DAUIN））；Rulex Innovation Labs（Rulex创新实验室）
An Explainable Market Integrity Monitoring System with Multi-Source Attention Signals and Transparent Scoring
Department of Computer Science University of New York（纽约大学计算机科学系）；DeepMind（深度Mind）；Mila, Université de Montréal（蒙特利尔大学Mila）；Google Research（谷歌研究）；CIFAR Fellow
German Cancer Consortium (DKTK)（德国癌症联盟（DKTK））；German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；Goethe University Frankfurt（法兰克福歌德大学）；Frankfurt Cancer Insititute（法兰克福癌症研究所）
Latent variable models are powerful statistical tools that can uncover relevant variation between patients or cells, by inferring unobserved hidden states from observable high-dimensional data. A major shortcoming of current methods, however, is their inability to learn sparse and interpretable hidden states. Additionally, in settings where partial knowledge on the latent structure of the data is readily available, a statistically sound integration of prior information into current methods is challenging. To address these issues, we propose spex-LVM, a factorial latent variable model with spar…
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Singapore Eye Research Institute（新加坡眼科研究所）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Institute for Molecular and Clinical Ophthalmology（分子与临床眼科研究所）；National University of Singapore（新加坡国立大学）
In this paper, we propose an explainable and interpretable diabetic retinopathy (ExplainDR) classification model based on neural-symbolic learning. To gain explainability, a highlevel symbolic representation should be considered in decision making. Specifically, we introduce a human-readable symbolic representation, which follows a taxonomy style of diabetic retinopathy characteristics related to eye health conditions to achieve explainability. We then include humanreadable features obtained from the symbolic representation in the disease prediction. Experimental results on a diabetic retinopa…
Developing Machine Learning (ML) algorithms for heterogeneous/mixed data is a longstanding problem. Many ML algorithms are not applicable to mixed data, which include numeric and non-numeric data, text, graphs and so on to generate interpretable models. Another longstanding problem is developing algorithms for lossless visualization of multidimensional mixed data. The further progress in ML heavily depends on success interpretable ML algorithms for mixed data and lossless interpretable visualization of multidimensional data. The later allows developing interpretable ML models using visual know…
ProtoTSNet: Interpretable Multivariate Time Series Classification With Prototypical Parts
Doctoral School of Exact and Natural Sciences Jagiellonian University（杰洛尼沃大学精确与自然科学博士学校）；Faculty of Physics, Astronomy and Applied Computer Science, Institute of Applied Computer Science, and Jagiellonian Human-Centered AI Lab Jagiellonian University（杰洛尼沃大学物理、天文学和应用计算机科学学院，应用计算机科学研究所，杰洛尼沃人本AI实验室）
Explainable AI Using Inherently Interpretable Components for Wearable-based Health Monitoring
Signal and System Theory Group, Paderborn University（信号与系统理论组，帕德博恩大学）；Department of Medicine I, Ludwig Maximilian University（医学部I，路德维希-马克西米利安大学）；Konrad Zuse School of Excellence in Reliable AI, Munich（可靠性人工智能卓越学校，慕尼黑）；Institute of Sports Medicine, Paderborn University（运动医学研究所，帕德博恩大学）；Division of Sports Neurology and Neurosciences, Mass General Brigham（运动神经学与神经科学部，马萨诸塞总医院布里奇顿分院）；Division of Epilepsy and Clinical Neurophysiology, Boston Children’s Hospital, Harvard Medical School（癫痫与临床神经生理学部，波士顿儿童医院，哈佛医学院）
Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction
Technical University of Darmstadt（达姆施塔特工业大学）；National Research Center for Applied Cybersecurity（国家应用网络安全研究中心）；Eindhoven University of Technology（埃因霍温理工大学）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；German Research Center for AI (DFKI)（德国人工智能研究中心）
KU Leuven（鲁汶大学）；University of Amsterdam（阿姆斯特丹大学）；Université Laval（拉瓦尔大学）
Highly regulated industries, like banking and insurance, ask for transparent decision-making algorithms. At the same time, competitive markets are pushing for the use of complex black box models. We therefore present a procedure to develop a Model-Agnostic Interpretable Data-driven suRRogate (maidrr) suited for structured tabular data. Knowledge is extracted from a black box via partial dependence effects. These are used to perform smart feature engineering by grouping variable values. This results in a segmentation of the feature space with automatic variable selection. A transparent generali…
Although modern machine learning and deep learning methods allow for complex and in-depth data analytics, the predictive models generated by these methods are often highly complex, and lack transparency. Explainable AI (XAI) methods are used to improve the interpretability of these complex models, and in doing so improve transparency. However, the inherent fitness of these explainable methods can be hard to evaluate. In particular, methods to evaluate the fidelity of the explanation to the underlying black box require further development, especially for tabular data. In this paper, we (a) prop…
For Artificial Intelligence to have a greater impact in biology and medicine, it is crucial that recommendations are both accurate and transparent. In other domains, a neurosymbolic approach of multi-hop reasoning on knowledge graphs has been shown to produce transparent explanations. However, there is a lack of research applying it to complex biomedical datasets and problems. In this paper, the approach is explored for drug discovery to draw solid conclusions on its applicability. For the first time, we systematically apply it to multiple biomedical datasets and recommendation tasks with fair…
Interpretable (not just posthoc-explainable) heterogeneous survivor bias-corrected treatment effects for assignment of postdischarge interventions to prevent readmissions
Mederrata（梅德拉塔）；National Institutes of Health（美国国立卫生研究院）；University of Vermont（佛蒙特大学）
John A. Paulson School of Engineering and Applied Sciences（约翰·A·保尔森工程与应用科学学院）；Harvard University（哈佛大学）
Machine learning models that first learn a representation of a domain in terms of human-understandable concepts, then use it to make predictions, have been proposed to facilitate interpretation and interaction with models trained on high-dimensional data. However these methods have important limitations: the way they define concepts are not inherently interpretable, and they assume that concept labels either exist for individual instances or can easily be acquired from users. These limitations are particularly acute for high-dimensional tabular features. We propose an approach for learning a s…
The Hong Kong University of Science and Technology（香港科技大学）
We enhance the accuracy and generalization of univariate time series point prediction by an explainable ensemble on the fly. We propose an Interpretable Dynamic Ensemble Architecture (IDEA), in which interpretable base learners give predictions independently with sparse communication as a group. The model is composed of several sequentially stacked groups connected by group backcast residuals and recurrent input competition. Ensemble driven by end-to-end training both horizontally and vertically brings state-of-the-art (SOTA) performances. Forecast accuracy improves by 2.6% over the best stati…
Electrocardiogram signals are omnipresent in medicine. A vital aspect in the analysis of this data is the identification and classification of heart beat types which is often done through automated algorithms. Advancements in neural networks and deep learning have led to a high classification accuracy. However, the final adoption of these models into clinical practice is limited due to the black-box nature of the methods. In this work, we explore the use of variational auto-encoders based on linear dense networks to learn human interpretable beat embeddings in time-series data. We demonstrate …
Massachusetts Institute of Technology（麻省理工学院）；Microsoft AI for Good Research Lab（微软AI向善研究实验室）；Rutherford Appleton Laboratory（卢瑟福·阿普尔顿实验室）；University of Reading（雷丁大学）
While the uptake of data-driven approaches for materials science and chemistry is at an exciting, early stage, to realise the true potential of machine learning models for successful scientific discovery, they must have qualities beyond purely predictive power. The predictions and inner workings of models should provide a certain degree of explainability by human experts, permitting the identification of potential model issues or limitations, building trust on model predictions and unveiling unexpected correlations that may lead to scientific insights. In this work, we summarize applications o…
From Explainable to Interpretable Deep Learning for Natural Language Processing in Healthcare: How Far from Reality?
School of Computer Science and Electronic Engineering, University of Essex（埃塞克斯大学计算机科学与电子工程学院）；Harvard Medical School（哈佛医学院）；Boston Children’s Hospital（波士顿儿童医院）；University of Southampton（南安普顿大学）；Athena Research Centre（雅典娜研究中心）
Explainable Artificial Intelligence Credit Risk Assessment using Machine Learning
Shreya Department of Computer Science and Engineering Specialisation in Data Science and Artificial Intelligence（Shreya 计算机科学与工程系 数据科学与人工智能专攻）；Mr. Harsh Pathak Assistant Professor Department of Computer Science and Engineering（Harsh Pathak 助理教授 计算机科学与工程系）；SRM University, Sonepat, Haryana, India（SRM 大学, Sonepat, Haryana, India）
Transparency in Machine Learning (ML), attempts to reveal the working mechanisms of complex models. Transparent ML promises to advance human factors engineering goals of human-centered AI in the target users. From a human-centered design perspective, transparency is not a property of the ML model but an affordance, i.e. a relationship between algorithm and user; as a result, iterative prototyping and evaluation with users is critical to attaining adequate solutions that afford transparency. However, following human-centered design principles in healthcare and medical image analysis is challeng…
Department of Artificial Intelligence, University of Malta（马耳他大学人工智能系）
With the ever-growing achievements in Artificial Intelligence (AI) and the recent boosted enthusiasm in Financial Technology (FinTech), applications such as credit scoring have gained substantial academic interest. Credit scoring helps financial experts make better decisions regarding whether or not to accept a loan application, such that loans with a high probability of default are not accepted. Apart from the noisy and highly imbalanced data challenges faced by such credit scoring models, recent regulations such as the `right to explanation' introduced by the General Data Protection Regulati…
