We consider the problem of explaining the predictions of graph neural networks (GNNs), which otherwise are considered as black boxes. Existing methods invariably focus on explaining the importance of graph nodes or edges but ignore the substructures of graphs, which are more intuitive and human-intelligible. In this work, we propose a novel method, known as SubgraphX, to explain GNNs by identifying important subgraphs. Given a trained GNN model and an input graph, our SubgraphX explains its predictions by efficiently exploring different subgraphs with Monte Carlo tree search. To make the tree search more effective, we propose to use Shapley values as a measure of subgraph importance, which can also capture the interactions among different subgraphs. To expedite computations, we propose efficient approximation schemes to compute Shapley values for graph data. Our work represents the first attempt to explain GNNs via identifying subgraphs explicitly and directly. Experimental results show that our SubgraphX achieves significantly improved explanations, while keeping computations at a reasonable level.
UiT the Arctic University of Norway（挪威北极大学）；Sapienza University of Rome（罗马第一大学）；NORCE Norwegian Research Centre（挪威NORCE研究中心）
Recently, subgraphs-enhanced Graph Neural Networks (SGNNs) have been introduced to enhance the expressive power of Graph Neural Networks (GNNs), which was proved to be not higher than the 1-dimensional Weisfeiler-Leman isomorphism test. The new paradigm suggests using subgraphs extracted from the input graph to improve the model's expressiveness, but the additional complexity exacerbates an already challenging problem in GNNs: explaining their predictions. In this work, we adapt PGExplainer, one of the most recent explainers for GNNs, to SGNNs. The proposed explainer accounts for the contribut…
GISExplainer: On Explainability of Graph Neural Networks via Game-theoretic Interaction Subgraphs
School of Cyber Security and Information Law, Chongqing University of Posts and Telecommunications（重庆邮电大学网络空间安全与信息法学院）；School of Computer Science and Technology, Chongqing University of Posts and Telecommunications（重庆邮电大学计算机科学与技术学院）；Chengdu University of Technology（成都理工大学）；School of Software Engineering, Chengdu University of Information Technology（成都信息工程大学软件工程学院）；School of Computer and Information Science, Chongqing Normal University（重庆师范大学计算机与信息科学学院）
Parallelizing Node-Level Explainability in Graph Neural Networks
Ericsson Cognitive Labs（爱立信认知实验室）；Institute for Research in Technology (IIT), ICAI School of Engineering, Comillas Pontifical University（技术研究所（IIT）、ICAI工程学院、科利马大学）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）
Subgraph recognition aims at discovering a compressed substructure of a graph that is most informative to the graph property. It can be formulated by optimizing Graph Information Bottleneck (GIB) with a mutual information estimator. However, GIB suffers from training instability and degenerated results due to its intrinsic optimization process. To tackle these issues, we reformulate the subgraph recognition problem into two steps: graph perturbation and subgraph selection, leading to a novel Variational Graph Information Bottleneck (VGIB) framework. VGIB first employs the noise injection to mo…
University of Southern California（南加州大学）；Jet Propulsion Laboratory（喷气推进实验室）；California Institute of Technology（加州理工学院）
Deep learning (DL) has proven to be an effective machine learning and computer vision technique. DL-based image segmentation, object recognition and classification will aid many in-situ Mars rover tasks such as path planning and artifact recognition/extraction. However, most of the Deep Neural Network (DNN) architectures are so complex that they are considered a 'black box'. In this paper, we used integrated gradients to describe the attributions of each neuron to the output classes. It provides a set of explainability tools (ET) that opens the black box of a DNN so that the individual contrib…
Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks
School of Computing and Information Sciences, Florida International University（佛罗里达国际大学计算与信息科学学院）；NEC Laboratories America（美国NEC实验室）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）
Backdoor attack is a powerful attack algorithm to deep learning model. Recently, GNN's vulnerability to backdoor attack has been proved especially on graph classification task. In this paper, we propose the first backdoor detection and defense method on GNN. Most backdoor attack depends on injecting small but influential trigger to the clean sample. For graph data, current backdoor attack focus on manipulating the graph structure to inject the trigger. We find that there are apparent differences between benign samples and malicious samples in some explanatory evaluation metrics, such as fideli…
Generating In-Distribution Proxy Graphs for Explaining Graph Neural Networks
Knight Foundation School of Computing and Information Sciences, Florida International University（佛罗里达国际大学奈特基金会计算与信息科学学院）；New Jersey Institute of Technology（新泽西理工学院）；University of Houston（休斯顿大学）；Visa Research（维萨研究中心）；Amazon Search A9（亚马逊搜索A9部门）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）
University of Science and Technology of China（中国科学技术大学）；National University of Singapore（新加坡国立大学）；Rice University（莱斯大学）
Explainability of graph neural networks (GNNs) aims to answer "Why the GNN made a certain prediction?", which is crucial to interpret the model prediction. The feature attribution framework distributes a GNN's prediction to its input features (e.g., edges), identifying an influential subgraph as the explanation. When evaluating the explanation (i.e., subgraph importance), a standard way is to audit the model prediction based on the subgraph solely. However, we argue that a distribution shift exists between the full graph and the subgraph, causing the out-of-distribution problem. Furthermore, w…
CoDy: Counterfactual Explainers for Dynamic Graphs
Centrum Wiskunde en Informatica, Amsterdam, Netherlands（荷兰阿姆斯特丹数学与信息学研究中心）；University of Amsterdam, Amsterdam, Netherlands（荷兰阿姆斯特丹大学）；Karlsruhe Institute of Technology, Karlsruhe, Germany（德国卡尔斯鲁厄技术大学）
University of Science and Technology of China（中国科学技术大学）；CCCD Key Lab of Ministry of Culture and Tourism（文化和旅游部CCCD重点实验室）；National University of Singapore（新加坡国立大学）
Explainability is crucial for probing graph neural networks (GNNs), answering questions like "Why the GNN model makes a certain prediction?". Feature attribution is a prevalent technique of highlighting the explanatory subgraph in the input graph, which plausibly leads the GNN model to make its prediction. Various attribution methods exploit gradient-like or attention scores as the attributions of edges, then select the salient edges with top attribution scores as the explanation. However, most of these works make an untenable assumption - the selected edges are linearly independent - thus lea…
GALAX: Graph-Augmented Language Model for Explainable Reinforcement-Guided Subgraph Reasoning in Precision Medicine
I2DB, Washington University School of Medicine（I2DB，华盛顿大学医学学院）；Department of Computer Science and Engineering, Washington University in St. Louis（计算机科学与工程系，华盛顿大学圣路易斯分校）；Department of Genetics, Washington University School of Medicine（遗传学系，华盛顿大学医学学院）
In this paper, we investigate the degree of explainability of graph neural networks (GNNs). Existing explainers work by finding global/local subgraphs to explain a prediction, but they are applied after a GNN has already been trained. Here, we propose a meta-learning framework for improving the level of explainability of a GNN directly at training time, by steering the optimization procedure towards what we call `interpretable minima'. Our framework (called MATE, MetA-Train to Explain) jointly trains a model to solve the original task, e.g., node classification, and to provide easily processab…
Unveiling and Steering Connectome Organization with Interpretable Latent Variables
National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（国家多媒体信息处理重点实验室，计算机科学学院，北京大学）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（计算机科学与工程学院，电子科学与技术大学）
The GECo algorithm for Graph Neural Networks Explanation
Dept of Mathematics and Computer Science University of Palermo（数学与计算机科学系巴勒莫大学）；Institute for High Performance Computing and Networking National Research Council of Italy（高性能计算与网络研究所意大利国家研究理事会）
Halıcıoğlu Data Science Institute（哈尔奇奥格卢数据科学研究所）；Universiy of California, San Diego（加州大学圣地亚哥分校）；Computer Science and Engineering（计算机科学与工程）；New York University（纽约大学）
Applied AI Institute, Deakin University（迪肯大学应用人工智能研究所）
We address a largely open problem of multilabel classification over graphs. Unlike traditional vector input, a graph has rich variable-size substructures which are related to the labels in some ways. We believe that uncovering these relations might hold the key to classification performance and explainability. We introduce GAML (Graph Attentional Multi-Label learning), a novel graph neural network that can handle this problem effectively. GAML regards labels as auxiliary nodes and models them in conjunction with the input graph. By applying message passing and attention mechanisms to both the …
SMT-EX: An Explainable Surrogate Modeling Toolbox for Mixed-Variables Design Exploration
AwanTunai（阿旺图奈）；ONERA（法国航空航天研究院）；Université de Toulouse（图卢兹大学）；Bandung Institute of Technology（万隆理工学院）；ISAE-SUPAERO（法国高等航空航天学院）；MINES ALBI（阿尔比矿业学院）；INSA（法国国立应用科学学院）；CNRS（法国国家科学研究中心）
Explanations as Bias Detectors: A Critical Study of Local Post-hoc XAI Methods for Fairness Exploration
University of Ioannina（伊奥安纳大学）；Archimedes / Athena RC Greece（阿基米德/亚特兰蒂斯研究所以及希腊）；Freie Universität Berlin（柏林自由大学）；Universität der Bundeswehr München（联邦国防军大学慕尼黑）
Recent years have witnessed rapid developments on collaborative filtering techniques for improving the performance of recommender systems due to the growing need of companies to help users discover new and relevant items. However, the majority of existing literature focuses on delivering items which match the user model learned from users' past preferences. A good recommendation model is expected to recommend items that are known to enjoy and items that are novel to try. In this work, we introduce an exploitation-exploration motivated variational auto-encoder (XploVAE) to collaborative filteri…
Recommendation systems personalise suggestions to individuals to help them in their decision making and exploration tasks. In the ideal case, these recommendations, besides of being accurate, should also be novel and explainable. However, up to now most platforms fail to provide both, novel recommendations that advance users' exploration along with explanations to make their reasoning more transparent to them. For instance, a well-known recommendation algorithm, such as matrix factorisation (MF), optimises only the accuracy criterion, while disregarding other quality criteria such as the expla…
University of Waterloo（滑铁卢大学）；Waterloo Artificial Intelligence Institute（滑铁卢人工智能研究所）；DarwinAI Corp.（达尔文人工智能公司）
There has been a significant surge of interest recently around the concept of explainable artificial intelligence (XAI), where the goal is to produce an interpretation for a decision made by a machine learning algorithm. Of particular interest is the interpretation of how deep neural networks make decisions, given the complexity and `black box' nature of such networks. Given the infancy of the field, there has been very limited exploration into the assessment of the performance of explainability methods, with most evaluations centered around subjective visual interpretation of the produced int…
Neural Graduated Assignment for Maximum Common Edge Subgraphs
School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Duke University（杜克大学）
We propose a new technique for performing state space exploration of closed loop control systems with neural network feedback controllers. Our approach involves approximating the sensitivity of the trajectories of the closed loop dynamics. Using such an approximator and the system simulator, we present a guided state space exploration method that can generate trajectories visiting the neighborhood of a target state at a specified time. We present a theoretical framework which establishes that our method will produce a sequence of trajectories that will reach a suitable neighborhood of the targ…
SPINEX-TimeSeries: Similarity-based Predictions with Explainable Neighbors Exploration for Time Series and Forecasting Problems
University of Manitoba（曼尼托巴大学）；Clemson University（克莱姆森大学）；Artificial Intelligence Research Institute for Science and Engineering (AIRISE), Clemson University（克莱姆森大学科学与工程人工智能研究院（AIRISE））
Federal Office for Information Security (BSI)（联邦信息安全办公室）
The nondeterminism of Deep Learning (DL) training algorithms and its influence on the explainability of neural network (NN) models are investigated in this work with the help of image classification examples. To discuss the issue, two convolutional neural networks (CNN) have been trained and their results compared. The comparison serves the exploration of the feasibility of creating deterministic, robust DL models and deterministic explainable artificial intelligence (XAI) in practice. Successes and limitation of all here carried out efforts are described in detail. The source code of the atta…
