Galanti, R., Coma-Puig, B., de Leoni, M., Carmona, J., Navarin, N.: Explainable Predictive Process Monitoring, the International Conference on Process Mining (ICPM 2020), IEEE Computational Intelligence Society, 2020. Article accepted for publication
Predictive Business Process Monitoring is becoming an essential aid for organizations, providing online operational support of their processes. This paper tackles the fundamental problem of equipping predictive business process monitoring with explanation capabilities, so that not only the what but also the why is reported when predicting generic KPIs like remaining time, or activity execution. We use the game theory of Shapley Values to obtain robust explanations of the predictions. The approach has been implemented and tested on real-life benchmarks, showing for the first time how explanations can be given in the field of predictive business process monitoring.
University of Southampton（南安普顿大学）；The Alexandra Institute（亚历山德拉研究所）
Monitoring machine learning models once they are deployed is challenging. It is even more challenging to decide when to retrain models in real-case scenarios when labeled data is beyond reach, and monitoring performance metrics becomes unfeasible. In this work, we use non-parametric bootstrapped uncertainty estimates and SHAP values to provide explainable uncertainty estimation as a technique that aims to monitor the deterioration of machine learning models in deployment environments, as well as determine the source of model deterioration when target labels are not available. Classical methods…
Predictive process analytics focuses on predicting the future states of running instances of a business process. While advanced machine learning techniques have been used to increase accuracy of predictions, the resulting predictive models lack transparency. Current explainable machine learning methods, such as LIME and SHAP, can be used to interpret black box models. However, it is unclear how fit for purpose these methods are in explaining process predictive models. In this paper, we draw on evaluation measures used in the field of explainable AI and propose functionally-grounded evaluation …
Predictive algorithms have a powerful potential to offer benefits in areas as varied as medicine or education. However, these algorithms and the data they use are built by humans, consequently, they can inherit the bias and prejudices present in humans. The outcomes can systematically repeat errors that create unfair results, which can even lead to situations of discrimination (e.g. gender, social or racial). In order to illustrate how important is to count with a diverse training dataset to avoid bias, we manipulate a well-known facial expression recognition dataset to explore gender bias and…
Predictive Process Analytics is becoming an essential aid for organizations, providing online operational support of their processes. However, process stakeholders need to be provided with an explanation of the reasons why a given process execution is predicted to behave in a certain way. Otherwise, they will be unlikely to trust the predictive monitoring technology and, hence, adopt it. This paper proposes a predictive analytics framework that is also equipped with explanation capabilities based on the game theory of Shapley Values. The framework has been implemented in the IBM Process Mining…
Meta-learning is a field that aims at discovering how different machine learning algorithms perform on a wide range of predictive tasks. Such knowledge speeds up the hyperparameter tuning or feature engineering. With the use of surrogate models various aspects of the predictive task such as meta-features, landmarker models e.t.c. are used to predict the expected performance. State of the art approaches are focused on searching for the best meta-model but do not explain how these different aspects contribute to its performance. However, to build a new generation of meta-models we need a deeper …
Predictive process monitoring is concerned with the analysis of events produced during the execution of a business process in order to predict as early as possible the final outcome of an ongoing case. Traditionally, predictive process monitoring methods are optimized with respect to accuracy. However, in environments where users make decisions and take actions in response to the predictions they receive, it is equally important to optimize the stability of the successive predictions made for each case. To this end, this paper defines a notion of temporal stability for binary classification ta…
Explainable AI in Grassland Monitoring: Enhancing Model Performance and Domain Adaptability
Leibniz-Institut für Agrartechnik und Bioökonomie e.V. (ATB)（莱布尼茨农业工程与生物经济研究所）；Technische Universität Berlin（柏林工业大学）；Understandable Machine Intelligence Lab (UMI Lab)（可理解机器智能实验室）；Universität Potsdam（波茨坦大学）
From Classification to Segmentation with Explainable AI: A Study on Crack Detection and Growth Monitoring
Intelligent Maintenance and Operations Systems (IMOS), EPFL（智能维护与运营系统研究所（IMOS），洛桑联邦理工学院）；Environmental Computational Science and Earth Observation (ECEO), EPFL（环境计算科学与地球观测研究所（ECEO），洛桑联邦理工学院）
While machine learning techniques have been successfully applied in several fields, the black-box nature of the models presents challenges for interpreting and explaining the results. We develop a new framework called Adaptive Explainable Neural Networks (AxNN) for achieving the dual goals of good predictive performance and model interpretability. For predictive performance, we build a structured neural network made up of ensembles of generalized additive model networks and additive index models (through explainable neural networks) using a two-stage process. This can be done using either a bo…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Saarland University（萨尔大学）
The contemporary process-aware information systems possess the capabilities to record the activities generated during the process execution. To leverage these process specific fine-granular data, process mining has recently emerged as a promising research discipline. As an important branch of process mining, predictive business process management, pursues the objective to generate forward-looking, predictive insights to shape business processes. In this study, we propose a conceptual framework sought to establish and promote understanding of decision-making environment, underlying business pro…
Feature selection of high-dimensional labeled data with limited observations is critical for making powerful predictive modeling accessible, scalable, and interpretable for domain experts. Spectroscopy data, which records the interaction between matter and electromagnetic radiation, particularly holds a lot of information in a single sample. Since acquiring such high-dimensional data is a complex task, it is crucial to exploit the best analytical tools to extract necessary information. In this paper, we investigate the most commonly used feature selection techniques and introduce applying rece…
An Explainable Market Integrity Monitoring System with Multi-Source Attention Signals and Transparent Scoring
Department of Computer Science University of New York（纽约大学计算机科学系）；DeepMind（深度Mind）；Mila, Université de Montréal（蒙特利尔大学Mila）；Google Research（谷歌研究）；CIFAR Fellow
Predictive process monitoring aims to predict future characteristics of an ongoing process case, such as case outcome or remaining timestamp. Recently, several predictive process monitoring methods based on deep learning such as Long Short-Term Memory or Convolutional Neural Network have been proposed to address the problem of next event prediction. However, due to insufficient training data or sub-optimal network configuration and architecture, these approaches do not generalize well the problem at hand. This paper proposes a novel adversarial training framework to address this shortcoming, b…
UiT The Arctic University of Norway（挪威北极大学）；Technical University of Berlin（柏林工业大学）
The need for interpretable models has fostered the development of self-explainable classifiers. Prior approaches are either based on multi-stage optimization schemes, impacting the predictive performance of the model, or produce explanations that are not transparent, trustworthy or do not capture the diversity of the data. To address these shortcomings, we propose ProtoVAE, a variational autoencoder-based framework that learns class-specific prototypes in an end-to-end manner and enforces trustworthiness and diversity by regularizing the representation space and introducing an orthonormality c…
An Explainable AI based approach for Monitoring Animal Health
Water Technology Development Hub(AWaDH), Indian Institute of Technology（印度理工学院水技术发展中心）；Academy of Scientific and Innovative Research (AcSIR)（科学与创新研究院）；Department of Higher Education（高等教育部）
The University of Hong Kong（香港大学）；Suoxinda Holdings Limited（索信达控股有限公司）
In recent years, the field of recommendation systems has attracted increasing attention to developing predictive models that provide explanations of why an item is recommended to a user. The explanations can be either obtained by post-hoc diagnostics after fitting a relatively complex model or embedded into an intrinsically interpretable model. In this paper, we propose the explainable recommendation systems based on a generalized additive model with manifest and latent interactions (GAMMLI). This model architecture is intrinsically interpretable, as it additively consists of the user and item…
Explainable AI Using Inherently Interpretable Components for Wearable-based Health Monitoring
Signal and System Theory Group, Paderborn University（信号与系统理论组，帕德博恩大学）；Department of Medicine I, Ludwig Maximilian University（医学部I，路德维希-马克西米利安大学）；Konrad Zuse School of Excellence in Reliable AI, Munich（可靠性人工智能卓越学校，慕尼黑）；Institute of Sports Medicine, Paderborn University（运动医学研究所，帕德博恩大学）；Division of Sports Neurology and Neurosciences, Mass General Brigham（运动神经学与神经科学部，马萨诸塞总医院布里奇顿分院）；Division of Epilepsy and Clinical Neurophysiology, Boston Children’s Hospital, Harvard Medical School（癫痫与临床神经生理学部，波士顿儿童医院，哈佛医学院）
paluno - The Ruhr Institute for Software Technology（帕鲁诺——鲁尔软件技术研究所）
Predictive business process monitoring increasingly leverages sophisticated prediction models. Although sophisticated models achieve consistently higher prediction accuracy than simple models, one major drawback is their lack of interpretability, which limits their adoption in practice. We thus see growing interest in explainable predictive business process monitoring, which aims to increase the interpretability of prediction models. Existing solutions focus on giving factual explanations.While factual explanations can be helpful, humans typically do not ask why a particular prediction was mad…
In this work, a machine learning approach is developed for predicting the outcomes of football matches. The novelty of this research lies in the utilisation of the Kelly Index to first classify matches into categories where each one denotes the different levels of predictive difficulty. Classification models using a wide suite of algorithms were developed for each category of matches in order to determine the efficacy of the approach. In conjunction to this, a set of previously unexplored features were engineering including Elo-based variables. The dataset originated from the Premier League ma…
We developed an explainable artificial intelligence (AI) early warning score (xAI-EWS) system for early detection of acute critical illness. While maintaining a high predictive performance, our system explains to the clinician on which relevant electronic health records (EHRs) data the prediction is grounded. Acute critical illness is often preceded by deterioration of routinely measured clinical parameters, e.g., blood pressure and heart rate. Early clinical prediction is typically based on manually calculated screening metrics that simply weigh these parameters, such as Early Warning Scores …
Several decision points exist in business processes (e.g., whether a purchase order needs a manager's approval or not), and different decisions are made for different process instances based on their characteristics (e.g., a purchase order higher than $500 needs a manager approval). Decision mining in process mining aims to describe/predict the routing of a process instance at a decision point of the process. By predicting the decision, one can take proactive actions to improve the process. For instance, when a bottleneck is developing in one of the possible decisions, one can predict the deci…
The Role of Explainable AI in Revolutionizing Human Health Monitoring: A Review
King Khalid University（哈立德国王大学）；Ummul Al Qura University（乌姆古拉大学）；Taif University（塔伊夫大学）；King Abdullah Medical City（阿卜杜拉国王医疗城）；University of Hafr Al Batin（哈夫尔巴廷大学）
Universidade de Santiago de Compostela（圣地亚哥-德孔波斯特拉大学）
Predictive monitoring of business processes is concerned with the prediction of ongoing cases on a business process. Lately, the popularity of deep learning techniques has propitiated an ever-growing set of approaches focused on predictive monitoring based on these techniques. However, the high disparity of process logs and experimental setups used to evaluate these approaches makes it especially difficult to make a fair comparison. Furthermore, it also difficults the selection of the most suitable approach to solve a specific problem. In this paper, we provide both a systematic literature rev…
Balancing Performance and Fairness in Explainable AI for Anomaly Detection in Distributed Power Plants Monitoring
African Institute for Mathematical Sciences (AIMS)（非洲数学科学研究所）；Rhodes University（罗德斯大学）；National Institute for Theoretical and Computational Sciences (NITheCS)（国家理论与计算科学研究所）；Ludwig Maximilian University of Munich（慕尼黑路易斯·马克西米利安大学）；University of Ferrara（费拉拉大学）
Explainable AI and Machine Learning for Exam-based Student Evaluation: Causal and Predictive Analysis of Socio-academic and Economic Factors
Institute of Information Technology Jahangirnagar University（贾汗吉尔纳加尔大学信息科技学院）；University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；Cefalo Dhaka Bangladesh（达卡 Bangladesh 的 Cefalo）
We propose a framework for interactive and explainable machine learning that enables users to (1) understand machine learning models; (2) diagnose model limitations using different explainable AI methods; as well as (3) refine and optimize the models. Our framework combines an iterative XAI pipeline with eight global monitoring and steering mechanisms, including quality monitoring, provenance tracking, model comparison, and trust building. To operationalize the framework, we present explAIner, a visual analytics system for interactive and explainable machine learning that instantiates all phas…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Saarland University（萨尔大学）
This study proposes an innovative explainable predictive quality analytics solution to facilitate data-driven decision-making for process planning in manufacturing by combining process mining, machine learning, and explainable artificial intelligence (XAI) methods. For this purpose, after integrating the top-floor and shop-floor data obtained from various enterprise information systems, a deep learning model was applied to predict the process outcomes. Since this study aims to operationalize the delivered predictive insights by embedding them into decision-making processes, it is essential to …
University of Lille（里尔大学）；ENSAIT（国立高等艺术与纺织工业学院）；GEMTEX–Laboratoire de Génie et Matériaux Textiles（纺织工程与材料实验室）；University of Kent（肯特大学）
Deep learning play a vital role in classifying different arrhythmias using the electrocardiography (ECG) data. Nevertheless, training deep learning models normally requires a large amount of data and it can lead to privacy concerns. Unfortunately, a large amount of healthcare data cannot be easily collected from a single silo. Additionally, deep learning models are like black-box, with no explainability of the predicted results, which is often required in clinical healthcare. This limits the application of deep learning in real-world health systems. In this paper, we design a new explainable a…
Explainable AI for Data-Driven Design of High-Dimensional Predictive Studies
School of Engineering, University of Edinburgh（爱丁堡大学工程学院）；Causality in Healthcare AI Hub (CHAI)（医疗因果AI枢纽）；Advanced Care Research Centre, Usher School of Population Health Sciences, University of Edinburgh（先进护理研究中心，乌瑟人口健康科学学院，爱丁堡大学）；Centre for Medical Informatics, Usher School of Population Health Sciences, University of Edinburgh（医学信息学中心，乌瑟人口健康科学学院，爱丁堡大学）
The field of predictive process monitoring focuses on case-level models to predict a single specific outcome such as a particular objective, (remaining) time, or next activity/remaining sequence. Recently, a longer-horizon, model-wide approach has been proposed in the form of process model forecasting, which predicts the future state of a whole process model through the forecasting of all activity-to-activity relations at once using time series forecasting. This paper introduces the concept of \emph{predictive process model monitoring} which sits in the middle of both predictive process monito…
RWTH Aachen University（亚琛工业大学）；Eindhoven University of Technology（埃因霍温理工大学）；Fraunhofer FIT（弗劳恩霍夫应用信息技术研究所）
Predictive process monitoring is a subfield of process mining that aims to estimate case or event features for running process instances. Such predictions are of significant interest to the process stakeholders. However, state-of-the-art methods for predictive monitoring require the training of complex machine learning models, which is often inefficient. This paper proposes an instance selection procedure that allows sampling training process instances for prediction models. We show that our sampling method allows for a significant increase of training speed for next activity prediction method…
Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS), Universidade de Santiago de Compostela（圣地亚哥-德孔波斯特拉大学智能技术研究中心（CiTIUS））；Universidade de Santiago de Compostela（圣地亚哥-德孔波斯特拉大学）；Departamento de Electrónica e Computación, Universidade de Santiago de Compostela（圣地亚哥-德孔波斯特拉大学电子与计算系）
Predictive monitoring of business processes is a subfield of process mining that aims to predict, among other things, the characteristics of the next event or the sequence of next events. Although multiple approaches based on deep learning have been proposed, mainly recurrent neural networks and convolutional neural networks, none of them really exploit the structural information available in process models. This paper proposes an approach based on graph convolutional networks and recurrent neural networks that uses information directly from the process model. An experimental evaluation on rea…
Università di Trieste（的里雅斯特大学）；Saarland University（萨尔大学）；Royal Holloway University（伦敦大学皇家霍洛威学院）
We consider the problem of predictive monitoring (PM), i.e., predicting at runtime future violations of a system from the current state. We work under the most realistic settings where only partial and noisy observations of the state are available at runtime. Such settings directly affect the accuracy and reliability of the reachability predictions, jeopardizing the safety of the system. In this work, we present a learning-based method for PM that produces accurate and reliable reachability predictions despite partial observability (PO). We build on Neural Predictive Monitoring (NPM), a PM met…
Northeastern University（东北大学）；NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心）；Bay Area Environmental Research Institute（湾区环境研究所）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
The El Niño Southern Oscillation (ENSO) is a semi-periodic fluctuation in sea surface temperature (SST) over the tropical central and eastern Pacific Ocean that influences interannual variability in regional hydrology across the world through long-range dependence or teleconnections. Recent research has demonstrated the value of Deep Learning (DL) methods for improving ENSO prediction as well as Complex Networks (CN) for understanding teleconnections. However, gaps in predictive understanding of ENSO-driven river flows include the black box nature of DL, the use of simple ENSO indices to descr…
Hierarchical Causal Abduction: A Foundation Framework for Explainable Model Predictive Control
System Dynamics, Chemnitz University of Technology, Chemnitz, Germany（化学特伦茨大学系统动力学系）；Department of Bioresources, Fraunhofer Institute for Molecular Biology and Applied Ecology（弗劳恩霍夫分子生物学与应用生态学研究所生物资源系）
