Glaucoma is the second driving reason for partial or complete blindness among all the visual deficiencies which mainly occurs because of excessive pressure in the eye due to anxiety or depression which damages the optic nerve and creates complications in vision. Traditional glaucoma screening is a time-consuming process that necessitates the medical professionals' constant attention, and even so time to time due to the time constrains and pressure they fail to classify correctly that leads to wrong treatment. Numerous efforts have been made to automate the entire glaucoma classification procedure however, these existing models in general have a black box characteristics that prevents users from understanding the key reasons behind the prediction and thus medical practitioners generally can not rely on these system. In this article after comparing with various pre-trained models, we propose a transfer learning model that is able to classify Glaucoma with 94.71\% accuracy. In addition, we have utilized Local Interpretable Model-Agnostic Explanations(LIME) that introduces explainability in our system. This improvement enables medical professionals obtain important and comprehensive information that aid them in making judgments. It also lessen the opacity and fragility of the traditional deep learning models.
A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME
William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London（伦敦玛丽女王大学威廉·哈维研究所、NIHR 巴茨生物医学研究中心）；Barts Heart Centre, St Bartholomew’s Hospital, Barts Health NHS Trust（巴茨心脏中心、圣巴塞洛缪医院、巴茨健康 NHS 信托）；Department of Population Health Sciences, University of Leicester（莱斯特大学人口健康科学系）；Department of Computer Science, Faculty of Science, University of Zakho（扎胡大学科学学院计算机科学系）；Department of Engineering for Innovation Medicine, University of Verona（维罗纳大学创新医学工程系）；Health Data Research UK（英国健康数据研究所）；Alan Turing Institute（艾伦·图灵研究所）
Predictive process analytics focuses on predicting the future states of running instances of a business process. While advanced machine learning techniques have been used to increase accuracy of predictions, the resulting predictive models lack transparency. Current explainable machine learning methods, such as LIME and SHAP, can be used to interpret black box models. However, it is unclear how fit for purpose these methods are in explaining process predictive models. In this paper, we draw on evaluation measures used in the field of explainable AI and propose functionally-grounded evaluation …
Explainable AI has attracted much research attention in recent years with feature attribution algorithms, which compute "feature importance" in predictions, becoming increasingly popular. However, there is little analysis of the validity of these algorithms as there is no "ground truth" in the existing datasets to validate their correctness. In this work, we develop a method to quantitatively evaluate the correctness of XAI algorithms by creating datasets with known explanation ground truth. To this end, we focus on the binary classification problems. String datasets are constructed using form…
AI to Identify Strain-sensitive Regions of the Optic Nerve Head Linked to Functional Loss in Glaucoma
National University of Singapore（新加坡国立大学）；Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Singapore Eye Research Institute（新加坡眼科研究所）；Singapore National Eye Centre（新加坡全国眼科中心）；Singapore-MIT Alliance for Research and Technology（新加坡-麻省理工学院科研联盟）；Massachusetts Institute of Technology（麻省理工学院）；Emory University School of Medicine（埃默里大学医学院）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
This paper applies eXplainable Artificial Intelligence (XAI) methods to investigate the socioeconomic disparities in COVID patient mortality. An Extreme Gradient Boosting (XGBoost) prediction model is built based on a de-identified Austin area hospital dataset to predict the mortality of COVID-19 patients. We apply two XAI methods, Shapley Additive exPlanations (SHAP) and Locally Interpretable Model Agnostic Explanations (LIME), to compare the global and local interpretation of feature importance. This paper demonstrates the advantages of using XAI which shows the feature importance and decisi…
On the Effectiveness of Methods and Metrics for Explainable AI in Remote Sensing Image Scene Classification
Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学学院，技术大学柏林）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究院）
Symbiosis Institute of Technology, Symbiosis International (Deemed University)（共生国际（被视为）大学共生技术学院）；Symbiosis Centre for Applied Artificial Intelligence, Symbiosis International (Deemed University)（共生国际（被视为）大学共生应用人工智能中心）；Princess Nourah Bint Abdulrahman University（努拉·宾特·阿卜杜勒拉赫曼公主大学）；Machine Intelligence Research Laboratories（机器智能研究实验室）
In this work, the integration of two machine learning approaches, namely domain adaptation and explainable AI, is proposed to address these two issues of generalized detection and explainability. Firstly the Domain Adversarial Neural Network (DANN) develops a generalized misinformation detector across multiple social media platforms DANN is employed to generate the classification results for test domains with relevant but unseen data. The DANN-based model, a traditional black-box model, cannot justify its outcome, i.e., the labels for the target domain. Hence a Local Interpretable Model-Agnost…
Which LIME should I trust? Concepts, Challenges, and Solutions
TU Clausthal（克劳斯塔尔工业大学）；University of Mannheim（曼海姆大学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Algorithmic approaches to interpreting machine learning models have proliferated in recent years. We carry out human subject tests that are the first of their kind to isolate the effect of algorithmic explanations on a key aspect of model interpretability, simulatability, while avoiding important confounding experimental factors. A model is simulatable when a person can predict its behavior on new inputs. Through two kinds of simulation tests involving text and tabular data, we evaluate five explanations methods: (1) LIME, (2) Anchor, (3) Decision Boundary, (4) a Prototype model, and (5) a Com…
University of Liverpool（利物浦大学）；Heriot-Watt University（赫瑞-瓦特大学）；Centrum voor Wiskunde en Informatica（荷兰国家数学与计算机科学研究所）；Delft University of Technology（代尔夫特理工大学）
Given the pressing need for assuring algorithmic transparency, Explainable AI (XAI) has emerged as one of the key areas of AI research. In this paper, we develop a novel Bayesian extension to the LIME framework, one of the most widely used approaches in XAI -- which we call BayLIME. Compared to LIME, BayLIME exploits prior knowledge and Bayesian reasoning to improve both the consistency in repeated explanations of a single prediction and the robustness to kernel settings. BayLIME also exhibits better explanation fidelity than the state-of-the-art (LIME, SHAP and GradCAM) by its ability to inte…
Explainable Severity ranking via pairwise n-hidden comparison: a case study of glaucoma
Information Sciences Institute, University of Southern California（南加州大学信息科学研究所）；College of Engineering and Computer Science, Vin University（文大学工程与计算机科学学院）
University of Western Australia（西澳大学）；Fiona Stanley Hospital（菲奥娜·斯坦利医院）；Harry Perkins Institute of Medical Research（哈里·珀金斯医学研究所）
Explainable Artificial Intelligence (XAI) has been identified as a viable method for determining the importance of features when making predictions using Machine Learning (ML) models. In this study, we created models that take an individual's health information (e.g. their drug history and comorbidities) as inputs, and predict the probability that the individual will have an Acute Coronary Syndrome (ACS) adverse outcome. Using XAI, we quantified the contribution that specific drugs had on these ACS predictions, thus creating an XAI-based technique for pharmacovigilance monitoring, using ACS as…
University of Engineering and Technology（工程技术大学）；Munster Technological University（芒斯特理工大学）；AJOU University（亚洲大学）
Prediction of a machine's Remaining Useful Life (RUL) is one of the key tasks in predictive maintenance. The task is treated as a regression problem where Machine Learning (ML) algorithms are used to predict the RUL of machine components. These ML algorithms are generally used as a black box with a total focus on the performance without identifying the potential causes behind the algorithms' decisions and their working mechanism. We believe, the performance (in terms of Mean Squared Error (MSE), etc.,) alone is not enough to build the trust of the stakeholders in ML prediction rather more insi…
Cybersecurity is a domain where the data distribution is constantly changing with attackers exploring newer patterns to attack cyber infrastructure. Intrusion detection system is one of the important layers in cyber safety in today's world. Machine learning based network intrusion detection systems started showing effective results in recent years. With deep learning models, detection rates of network intrusion detection system are improved. More accurate the model, more the complexity and hence less the interpretability. Deep neural networks are complex and hard to interpret which makes diffi…
German Research Center for Artificial Intelligence (DFKI GmbH)（德国人工智能研究中心 (DFKI GmbH)）；TU Kaiserslautern（凯泽斯劳滕工业大学）
The field of explainable AI (XAI) has quickly become a thriving and prolific community. However, a silent, recurrent and acknowledged issue in this area is the lack of consensus regarding its terminology. In particular, each new contribution seems to rely on its own (and often intuitive) version of terms like "explanation" and "interpretation". Such disarray encumbers the consolidation of advances in the field towards the fulfillment of scientific and regulatory demands e.g., when comparing methods or establishing their compliance with respect to biases and fairness constraints. We propose a t…
Aalto University（阿尔托大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Umeå University（于默奥大学）
Machine learning-based systems are rapidly gaining popularity and in-line with that there has been a huge research surge in the field of explainability to ensure that machine learning models are reliable, fair, and can be held liable for their decision-making process. Explainable Artificial Intelligence (XAI) methods are typically deployed to debug black-box machine learning models but in comparison to tabular, text, and image data, explainability in time series is still relatively unexplored. The aim of this study was to achieve and evaluate model agnostic explainability in a time series fore…
Kalasalingam Academy for Research and Education（卡萨斯林甘研究与教育学院）；GE Corporate Research Center（通用电气企业研究中心）；SINTEF；Norwegian University of Science and Technology（挪威科技大学）；SUNY Polytechnic Institute（纽约州立大学理工学院）
This paper presents an eXplainable Fault Detection and Diagnosis System (XFDDS) for incipient faults in PV panels. The XFDDS is a hybrid approach that combines the model-based and data-driven framework. Model-based FDD for PV panels lacks high fidelity models at low irradiance conditions for detecting incipient faults. To overcome this, a novel irradiance based three diode model (IB3DM) is proposed. It is a nine parameter model that provides higher accuracy even at low irradiance conditions, an important aspect for detecting incipient faults from noise. To exploit PV data, extreme gradient boo…
Opening the black box of deep learning: Validating the statistical association between explainable artificial intelligence (XAI) and clinical domain knowledge in fundus image-based glaucoma diagnosis
Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Singapore National Eye Centre（新加坡国家眼科中心）
Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所）
Not only automation of manufacturing processes but also automation of automation procedures itself become increasingly relevant to automation research. In this context, automated capability assessment, mainly leveraged by deep learning systems driven from 3D CAD data, have been presented. Current assessment systems may be able to assess CAD data with regards to abstract features, e.g. the ability to automatically separate components from bulk goods, or the presence of gripping surfaces. Nevertheless, they suffer from the factor of black box systems, where an assessment can be learned and gener…
Parkinson's disease (PD) is a degenerative and progressive neurological condition. Early diagnosis can improve treatment for patients and is performed through dopaminergic imaging techniques like the SPECT DaTscan. In this study, we propose a machine learning model that accurately classifies any given DaTscan as having Parkinson's disease or not, in addition to providing a plausible reason for the prediction. This is kind of reasoning is done through the use of visual indicators generated using Local Interpretable Model-Agnostic Explainer (LIME) methods. DaTscans were drawn from the Parkinson'…
Financial Fraud Detection Using Explainable AI and Stacking Ensemble Methods
Computer Science Department, College of Computers and Information Technology, Taif University（计算机科学系，计算机与信息科技学院，泰夫大学）；Department of Computer Science, Taif University（计算机科学系，泰夫大学）
In this study, we presented a four-level cross-domain strategy for fake news detection on pre-trained models. Cross-domain text classification is a task of a model adopting a target domain by using the knowledge of the source domain. Explainability is crucial in understanding the behaviour of these complex models. A fine-tune BERT model is used to. perform cross-domain classification with several experiments using datasets from different domains. Explanatory models like Anchor, ELI5, LIME and SHAP are used to design a novel explainable approach to cross-domain levels. The experimental analysis…
Aalto University（阿尔托大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
In the present paper we present the potential of Explainable Artificial Intelligence methods for decision-support in medical image analysis scenarios. With three types of explainable methods applied to the same medical image data set our aim was to improve the comprehensibility of the decisions provided by the Convolutional Neural Network (CNN). The visual explanations were provided on in-vivo gastral images obtained from a Video capsule endoscopy (VCE), with the goal of increasing the health professionals' trust in the black box predictions. We implemented two post-hoc interpretable machine l…
Breast Cancer Diagnosis: A Comprehensive Exploration of Explainable Artificial Intelligence (XAI) Techniques
Canadian Institute of Cybersecurity (CIC)（加拿大网络安全研究所）；University of New Brunswick（新不伦瑞克大学）；Dipartimento di Informatica, Università di Verona（威尼斯大学信息学院）；Università di Verona（威尼斯大学）；Department of Computer Science, School of Mathematics and Computer Science, Institute of Business Administration（计算机科学系，数学与计算机科学学院，商学院）；LIRIS, CNRS, Université Claude Bernard Lyon 1（LIRIS，国家科学研究中心，克莱尔伯恩大学 Lyon 1 分校）；LHC, UMR5516, Université de Saint-Etienne（LHC，UMR5516，圣艾蒂安大学）
A plethora of recent research has proposed several automated methods based on machine learning (ML) and deep learning (DL) to detect cybersickness in Virtual reality (VR). However, these detection methods are perceived as computationally intensive and black-box methods. Thus, those techniques are neither trustworthy nor practical for deploying on standalone VR head-mounted displays (HMDs). This work presents an explainable artificial intelligence (XAI)-based framework VR-LENS for developing cybersickness detection ML models, explaining them, reducing their size, and deploying them in a Qualcom…
KU Leuven（鲁汶大学）；UZ Leuven（鲁汶大学医院）；University of Porto（波尔图大学）；Centro Hospitalar e Universitário São João（圣若昂大学医院中心）；Hasselt University（哈瑟尔特大学）；University of Antwerp（安特卫普大学）；Flemish Institute for Technological Research（弗拉芒技术研究院）
Although unprecedented sensitivity and specificity values are reported, recent glaucoma detection deep learning models lack in decision transparency. Here, we propose a methodology that advances explainable deep learning in the field of glaucoma detection and vertical cup-disc ratio (VCDR), an important risk factor. We trained and evaluated deep learning models using fundus images that underwent a certain cropping policy. We defined the crop radius as a percentage of image size, centered on the optic nerve head (ONH), with an equidistant spaced range from 10%-60% (ONH crop policy). The inverse…
