In recent years, the field of recommendation systems has attracted increasing attention to developing predictive models that provide explanations of why an item is recommended to a user. The explanations can be either obtained by post-hoc diagnostics after fitting a relatively complex model or embedded into an intrinsically interpretable model. In this paper, we propose the explainable recommendation systems based on a generalized additive model with manifest and latent interactions (GAMMLI). This model architecture is intrinsically interpretable, as it additively consists of the user and item main effects, the manifest user-item interactions based on observed features, and the latent interaction effects from residuals. Unlike conventional collaborative filtering methods, the group effect of users and items are considered in GAMMLI. It is beneficial for enhancing the model interpretability, and can also facilitate the cold-start recommendation problem. A new Python package GAMMLI is developed for efficient model training and visualized interpretation of the results. By numerical experiments based on simulation data and real-world cases, the proposed method is shown to have advantages in both predictive performance and explainable recommendation.
While machine learning techniques have been successfully applied in several fields, the black-box nature of the models presents challenges for interpreting and explaining the results. We develop a new framework called Adaptive Explainable Neural Networks (AxNN) for achieving the dual goals of good predictive performance and model interpretability. For predictive performance, we build a structured neural network made up of ensembles of generalized additive model networks and additive index models (through explainable neural networks) using a two-stage process. This can be done using either a bo…
The University of Hong Kong（香港大学）；Wells Fargo（富国银行）
The lack of interpretability is an inevitable problem when using neural network models in real applications. In this paper, an explainable neural network based on generalized additive models with structured interactions (GAMI-Net) is proposed to pursue a good balance between prediction accuracy and model interpretability. GAMI-Net is a disentangled feedforward network with multiple additive subnetworks; each subnetwork consists of multiple hidden layers and is designed for capturing one main effect or one pairwise interaction. Three interpretability aspects are further considered, including a)…
Financial Fraud Detection Using Explainable AI and Stacking Ensemble Methods
Computer Science Department, College of Computers and Information Technology, Taif University（计算机科学系，计算机与信息科技学院，泰夫大学）；Department of Computer Science, Taif University（计算机科学系，泰夫大学）
Machine Learning algorithms are increasingly being used in recent years due to their flexibility in model fitting and increased predictive performance. However, the complexity of the models makes them hard for the data analyst to interpret the results and explain them without additional tools. This has led to much research in developing various approaches to understand the model behavior. In this paper, we present the Explainable Neural Network (xNN), a structured neural network designed especially to learn interpretable features. Unlike fully connected neural networks, the features engineered…
Employing Explainable Artificial Intelligence (XAI) Methodologies to Analyze the Correlation between Input Variables and Tensile Strength in Additively Manufactured Samples
School of Industrial and Information Engineering, Politecnico Di Milano（米兰理工大学工业与信息工程学院）；Symbiosis Institute of Technology, Symbiosis International (Deemed) University（Symbiosis International (Deemed) University Symbiosis Institute of Technology）
FlagGAM: Rule-Basis Generalized Additive Models for Explainable Tabular Prediction
EECS Department, Massachusetts Institute of Technology, Cambridge, MA, USA（麻省理工学院电子工程与计算机科学系）；Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA（麻省理工学院斯隆管理学院）
University of Tsukuba（筑波大学）；NTT DATA Corporation（NTT数据公司）
Transparency of Machine Learning models used for decision support in various industries becomes essential for ensuring their ethical use. To that end, feature attribution methods such as SHAP (SHapley Additive exPlanations) are widely used to explain the predictions of black-box machine learning models to customers and developers. However, a parallel trend has been to train machine learning models in collaboration with other data holders without accessing their data. Such models, trained over horizontally or vertically partitioned data, present a challenge for explainable AI because the explai…
Warsaw University of Technology（华沙理工大学）；University of Warsaw（华沙大学）
Explainable Artificial Intelligence (XAI)has received a great deal of attention recently. Explainability is being presented as a remedy for the distrust of complex and opaque models. Model agnostic methods such as LIME, SHAP, or Break Down promise instance-level interpretability for any complex machine learning model. But how faithful are these additive explanations? Can we rely on additive explanations for non-additive models? In this paper, we (1) examine the behavior of the most popular instance-level explanations under the presence of interactions, (2) introduce a new method that detects i…
By computing the rank correlation between attention weights and feature-additive explanation methods, previous analyses either invalidate or support the role of attention-based explanations as a faithful and plausible measure of salience. To investigate whether this approach is appropriate, we compare LIME, Integrated Gradients, DeepLIFT, Grad-SHAP, Deep-SHAP, and attention-based explanations, applied to two neural architectures trained on single- and pair-sequence language tasks. In most cases, we find that none of our chosen methods agree. Based on our empirical observations and theoretical …
University of Engineering and Technology（工程技术大学）；Munster Technological University（芒斯特理工大学）；AJOU University（亚洲大学）
Prediction of a machine's Remaining Useful Life (RUL) is one of the key tasks in predictive maintenance. The task is treated as a regression problem where Machine Learning (ML) algorithms are used to predict the RUL of machine components. These ML algorithms are generally used as a black box with a total focus on the performance without identifying the potential causes behind the algorithms' decisions and their working mechanism. We believe, the performance (in terms of Mean Squared Error (MSE), etc.,) alone is not enough to build the trust of the stakeholders in ML prediction rather more insi…
University of Tübingen（图宾根大学）；Tübingen AI Center（图宾根人工智能中心）
In explainable machine learning, local post-hoc explanation algorithms and inherently interpretable models are often seen as competing approaches. This work offers a partial reconciliation between the two by establishing a correspondence between Shapley Values and Generalized Additive Models (GAMs). We introduce $n$-Shapley Values, a parametric family of local post-hoc explanation algorithms that explain individual predictions with interaction terms up to order $n$. By varying the parameter $n$, we obtain a sequence of explanations that covers the entire range from Shapley Values up to a uniqu…
KASTEL Security Research Labs（KASTEL安全研究实验室）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Explainable machine learning holds great potential for analyzing and understanding learning-based systems. These methods can, however, be manipulated to present unfaithful explanations, giving rise to powerful and stealthy adversaries. In this paper, we demonstrate blinding attacks that can fully disguise an ongoing attack against the machine learning model. Similar to neural backdoors, we modify the model's prediction upon trigger presence but simultaneously also fool the provided explanation. This enables an adversary to hide the presence of the trigger or point the explanation to entirely d…
Robustness of Explainable Artificial Intelligence in Industrial Process Modelling
Technical University Graz（格拉茨工业大学）；voestalpine Stahl GmbH（奥钢联钢铁股份公司）；K1-MET GmbH（K1-MET有限公司）；Christian Doppler Laboratory for Dependable Intelligent Systems in Harsh Environments（克里斯蒂安·多普勒恶劣环境可靠智能系统实验室）
São Paulo State University - UNESP（圣保罗州立大学（UNESP））
Automatic Text Summarization (ATS) is becoming relevant with the growth of textual data; however, with the popularization of public large-scale datasets, some recent machine learning approaches have focused on dense models and architectures that, despite producing notable results, usually turn out in models difficult to interpret. Given the challenge behind interpretable learning-based text summarization and the importance it may have for evolving the current state of the ATS field, this work studies the application of two modern Generalized Additive Models with interactions, namely Explainabl…
A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME
William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London（伦敦玛丽女王大学威廉·哈维研究所、NIHR 巴茨生物医学研究中心）；Barts Heart Centre, St Bartholomew’s Hospital, Barts Health NHS Trust（巴茨心脏中心、圣巴塞洛缪医院、巴茨健康 NHS 信托）；Department of Population Health Sciences, University of Leicester（莱斯特大学人口健康科学系）；Department of Computer Science, Faculty of Science, University of Zakho（扎胡大学科学学院计算机科学系）；Department of Engineering for Innovation Medicine, University of Verona（维罗纳大学创新医学工程系）；Health Data Research UK（英国健康数据研究所）；Alan Turing Institute（艾伦·图灵研究所）
A Unified Framework for Uncertainty-Aware Explainable Artificial Intelligence: A Case Study in Power Quality Disturbance Classification
School of Engineering, Deakin University（德肯大学工程学院）；Faculty of Mathematics and Computer Science, FernUniversität in Hagen（哈根应用科学大学数学与计算机科学学院）；Department of Computing Technologies, Swinburne University of Technology（斯威本科技大学计算技术学院）
The transformation of conventional power networks into smart grids with the heavy penetration level of renewable energy resources, particularly grid-connected Photovoltaic (PV) systems, has increased the need for efficient fault identification systems. Malfunctioning any single component in grid-connected PV systems may lead to grid instability and other serious consequences, showing that a reliable fault identification system is the utmost requirement for ensuring operational integrity. Therefore, this paper presents a novel fault identification approach based on statistical signatures of PV …
Interpretability of learning-to-rank models is a crucial yet relatively under-examined research area. Recent progress on interpretable ranking models largely focuses on generating post-hoc explanations for existing black-box ranking models, whereas the alternative option of building an intrinsically interpretable ranking model with transparent and self-explainable structure remains unexplored. Developing fully-understandable ranking models is necessary in some scenarios (e.g., due to legal or policy constraints) where post-hoc methods cannot provide sufficiently accurate explanations. In this …
A plethora of recent research has proposed several automated methods based on machine learning (ML) and deep learning (DL) to detect cybersickness in Virtual reality (VR). However, these detection methods are perceived as computationally intensive and black-box methods. Thus, those techniques are neither trustworthy nor practical for deploying on standalone VR head-mounted displays (HMDs). This work presents an explainable artificial intelligence (XAI)-based framework VR-LENS for developing cybersickness detection ML models, explaining them, reducing their size, and deploying them in a Qualcom…
School of Engineering（工程学院）；Zurich University of Applied Sciences（苏黎世应用科技大学）；Data Science Institute（数据科学研究所）；Columbia University（哥伦比亚大学）
Artificial Intelligence (AI) has created the single biggest technology revolution the world has ever seen. For the finance sector, it provides great opportunities to enhance customer experience, democratize financial services, ensure consumer protection and significantly improve risk management. While it is easier than ever to run state-of-the-art machine learning models, designing and implementing systems that support real-world finance applications have been challenging. In large part because they lack transparency and explainability which are important factors in establishing reliable techn…
