Graph kernels are historically the most widely-used technique for graph classification tasks. However, these methods suffer from limited performance because of the hand-crafted combinatorial features of graphs. In recent years, graph neural networks (GNNs) have become the state-of-the-art method in downstream graph-related tasks due to their superior performance. Most GNNs are based on Message Passing Neural Network (MPNN) frameworks. However, recent studies show that MPNNs can not exceed the power of the Weisfeiler-Lehman (WL) algorithm in graph isomorphism test. To address the limitations of existing graph kernel and GNN methods, in this paper, we propose a novel GNN framework, termed \textit{Kernel Graph Neural Networks} (KerGNNs), which integrates graph kernels into the message passing process of GNNs. Inspired by convolution filters in convolutional neural networks (CNNs), KerGNNs adopt trainable hidden graphs as graph filters which are combined with subgraphs to update node embeddings using graph kernels. In addition, we show that MPNNs can be viewed as special cases of KerGNNs. We apply KerGNNs to multiple graph-related tasks and use cross-validation to make fair comparisons with benchmarks. We show that our method achieves competitive performance compared with existing state-of-the-art methods, demonstrating the potential to increase the representation ability of GNNs. We also show that the trained graph filters in KerGNNs can reveal the local graph structures of the dataset, which significantly improves the model interpretability compared with conventional GNN models.
Eskişehir Teknik Üniversitesi（埃斯基谢希尔技术大学）；Erasmus University Rotterdam（鹿特丹伊拉斯姆斯大学）
Kernels are often developed and used as implicit mapping functions that show impressive predictive power due to their high-dimensional feature space representations. In this study, we gradually construct a series of simple feature maps that lead to a collection of interpretable low-dimensional kernels. At each step, we keep the original features and make sure that the increase in the dimension of input data is extremely low, so that the resulting discriminant functions remain interpretable and amenable to fast training. Despite our persistence on interpretability, we obtain high accuracy resul…
Center of Cognitive Interactive Technology (CITEC)（认知交互技术中心（CITEC））；Bielefeld University（比勒费尔德大学）
Prototype-based methods are of the particular interest for domain specialists and practitioners as they summarize a dataset by a small set of representatives. Therefore, in a classification setting, interpretability of the prototypes is as significant as the prediction accuracy of the algorithm. Nevertheless, the state-of-the-art methods make inefficient trade-offs between these concerns by sacrificing one in favor of the other, especially if the given data has a kernel-based representation. In this paper, we propose a novel interpretable multiple-kernel prototype learning (IMKPL) to construct…
Sparse learned kernels for interpretable and efficient medical time series processing
Duke University School of Medicine（杜克大学医学院）；Duke University（杜克大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Nell Hodgson Woodruff School of Nursing, Emory University（埃默里大学内尔·霍奇森·伍德拉夫护理学院）
Kernel dimensionality reduction (KDR) algorithms find a low dimensional representation of the original data by optimizing kernel dependency measures that are capable of capturing nonlinear relationships. The standard strategy is to first map the data into a high dimensional feature space using kernels prior to a projection onto a low dimensional space. While KDR methods can be easily solved by keeping the most dominant eigenvectors of the kernel matrix, its features are no longer easy to interpret. Alternatively, Interpretable KDR (IKDR) is different in that it projects onto a subspace \textit…
RFFNet: Large-Scale Interpretable Kernel Methods via Random Fourier Features
Federal University of São Carlos（圣卡洛斯联邦大学）；Universidade de São Paulo（圣保罗大学）；Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo（圣保罗大学数学与计算机科学学院）
Data-Driven Integration Kernels for Interpretable Nonlocal Operator Learning
Department of Earth System Science（地球系统科学系）；University of California, Irvine（加州大学伊维福分校）；Department of Computing and Data Science（计算与数据科学系）；Boston University（波士顿大学）；The Center for Atmosphere Ocean Science（大气海洋科学中心）；New York University（纽约大学）；NVIDIA Corporation（NVIDIA公司）；Faculty of Geosciences and Environment（地球科学与环境学院）；University of Lausanne（洛桑大学）；Expertise Center for Climate Extremes（极端气候专家中心）
In recent years, deep learning has shown potential and efficiency in a wide area including computer vision, image and signal processing. Yet, translational challenges remain for user applications due to a lack of interpretability of algorithmic decisions and results. This black box problem is particularly problematic for high-risk applications such as medical-related decision-making. The current study goal was to design an interpretable deep learning system for time series classification of electroencephalogram (EEG) for sleep stage scoring as a step toward designing a transparent system. We h…
Current neural architecture search (NAS) strategies focus only on finding a single, good, architecture. They offer little insight into why a specific network is performing well, or how we should modify the architecture if we want further improvements. We propose a Bayesian optimisation (BO) approach for NAS that combines the Weisfeiler-Lehman graph kernel with a Gaussian process surrogate. Our method optimises the architecture in a highly data-efficient manner: it is capable of capturing the topological structures of the architectures and is scalable to large graphs, thus making the high-dimen…
Mode decomposition is a prototypical pattern recognition problem that can be addressed from the (a priori distinct) perspectives of numerical approximation, statistical inference and deep learning. Could its analysis through these combined perspectives be used as a Rosetta stone for deciphering mechanisms at play in deep learning? Motivated by this question we introduce programmable and interpretable regression networks for pattern recognition and address mode decomposition as a prototypical problem. The programming of these networks is achieved by assembling elementary modules decomposing and…
Oden Institute for Computational Engineering and Sciences（奥登计算工程与科学研究所）；Department of Computer Science（计算机科学系）；School of Information（信息学院）；University of Texas, Austin（得克萨斯大学奥斯汀分校）
Partial Differential Equations are infinite dimensional encoded representations of physical processes. However, imbibing multiple observation data towards a coupled representation presents significant challenges. We present a fully convolutional architecture that captures the invariant structure of the domain to reconstruct the observable system. The proposed architecture is significantly low-weight compared to other networks for such problems. Our intent is to learn coupled dynamic processes interpreted as deviations from true kernels representing isolated processes for model-adaptivity. Expe…
Saarland Informatics Campus（萨尔信息学园区）；University of Tübingen（图宾根大学）
Due to the complexity of cancer, clustering algorithms have been used to disentangle the observed heterogeneity and identify cancer subtypes that can be treated specifically. While kernel based clustering approaches allow the use of more than one input matrix, which is an important factor when considering a multidimensional disease like cancer, the clustering results remain hard to evaluate and, in many cases, it is unclear which piece of information had which impact on the final result. In this paper, we propose an extension of multiple kernel learning clustering that enables the characteriza…
Fast, accurate, and interpretable decoding of electrocorticographic signals using dynamic mode decomposition
Osaka University（大阪大学）；Osaka University Graduate School of Medicine（大阪大学大学院医学系研究科）；Institute for Quantum Life Science, National Institutes for Quantum Science and Technology（量子科学技术研究开发机构量子生命科学研究所）；RIKEN Center for Advanced Intelligence Project（理化学研究所智能研究中心）；ATR Neural Information Analysis Laboratories（ATR神经信息分析实验室）
Deep Gaussian Processes (DGPs) combine the expressiveness of Deep Neural Networks (DNNs) with quantified uncertainty of Gaussian Processes (GPs). Expressive power and intractable inference both result from the non-Gaussian distribution over composition functions. We propose interpretable DGP based on approximating DGP as a GP by calculating the exact moments, which additionally identify the heavy-tailed nature of some DGP distributions. Consequently, our approach admits interpretation as both NNs with specified activation functions and as a variational approximation to DGP. We identify the exp…
Interpretable Graph-Level Anomaly Detection via Contrast with Normal Prototypes
State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；School of Artificial Intelligence, Nanjing University（人工智能学院，南京大学）
NEC Corporation（日本电气股份有限公司）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
The interpretability of machine learning, particularly for deep neural networks, is crucial for decision making in real-world applications. One approach is replacing the un-interpretable machine learning model with a surrogate model, which has a simple structure for interpretation. Another approach is understanding the target system by using a simulation modeled by human knowledge with interpretable simulation parameters. Recently, simulator calibration has been developed based on kernel mean embedding to estimate the simulation parameters as posterior distributions. Our idea is to use a simul…
CRITS: Convolutional Rectifier for Interpretable Time Series Classification
Department of Computer and Systems Sciences, Stockholm University（斯德哥尔摩大学计算机与系统科学系）；Department of Computer Science and Engineering, University of Oviedo（奥维多大学计算机科学与工程系）
Georgetown University（乔治城大学）；Meng Cheng Family Medicine Clinic（孟诚家庭医学科诊所）；Tri-Service General Hospital（三军总医院）；Fu Jen Catholic University（辅仁大学）
We present an end-to-end, interpretable, deep-learning architecture to learn a graph kernel that predicts the outcome of chronic disease drug prescription. This is achieved through a deep metric learning collaborative with a Support Vector Machine objective using a graphical representation of Electronic Health Records. We formulate the predictive model as a binary graph classification problem with an adaptive learned graph kernel through novel cross-global attention node matching between patient graphs, simultaneously computing on multiple graphs without training pair or triplet generation. Re…
Microsoft Research（微软研究院）；Indian Institute of Science（印度科学院）；Microsoft（微软）
The state-of-the-art CNN models give good performance on sentence classification tasks. The purpose of this work is to empirically study desirable properties such as semantic coherence, attention mechanism and reusability of CNNs in these tasks. Semantically coherent kernels are preferable as they are a lot more interpretable for explaining the decision of the learned CNN model. We observe that the learned kernels do not have semantic coherence. Motivated by this observation, we propose to learn kernels with semantic coherence using clustering scheme combined with Word2Vec representation and d…
University of Pittsburgh Medical Center（匹兹堡大学医学中心）；University of Pittsburgh（匹兹堡大学）；Weill Cornell Medicine（威尔康奈尔医学院）
We consider the problem of interpretable network representation learning for samples of network-valued data. We propose the Principal Component Analysis for Networks (PCAN) algorithm to identify statistically meaningful low-dimensional representations of a network sample via subgraph count statistics. The PCAN procedure provides an interpretable framework for which one can readily visualize, explore, and formulate predictive models for network samples. We furthermore introduce a fast sampling-based algorithm, sPCAN, which is significantly more computationally efficient than its counterpart, bu…
Kernel Regression of Multi-Way Data via Tensor Trains with Hadamard Overparametrization: The Dynamic Graph Flow Case
Department of Information and Communications Engineering, Institute of Science Tokyo, Japan（信息与通信工程系，东京科学研究所，日本）；Department of Statistics and Insurance Science, University of Piraeus, Greece（统计与保险科学系，比雷埃克斯大学，希腊）；CA-AI, Department of Electrical Engineering and Computer Science, Florida Atlantic University, USA（CA-AI，电气工程与计算机科学系，佛罗里达亚特兰大大学，美国）
