Recent work in graph neural networks (GNNs) has led to improvements in molecular activity and property prediction tasks. Unfortunately, GNNs often fail to capture the relative importance of interactions between molecular substructures, in part due to the absence of efficient intermediate pooling steps. To address these issues, we propose LaPool (Laplacian Pooling), a novel, data-driven, and interpretable hierarchical graph pooling method that takes into account both node features and graph structure to improve molecular representation. We benchmark LaPool on molecular graph prediction and understanding tasks and show that it outperforms recent GNNs. Interestingly, LaPool also remains competitive on non-molecular tasks. Both quantitative and qualitative assessments are done to demonstrate LaPool's improved interpretability and highlight its potential benefits in drug design. Finally, we demonstrate LaPool's utility for the generation of valid and novel molecules by incorporating it into an adversarial autoencoder.
University of Pittsburgh（匹兹堡大学）；Intel Labs（英特尔实验室）；Lehigh University（里海大学）
Recent years have witnessed the emergence and flourishing of hierarchical graph pooling neural networks (HGPNNs) which are effective graph representation learning approaches for graph level tasks such as graph classification. However, current HGPNNs do not take full advantage of the graph's intrinsic structures (e.g., community structure). Moreover, the pooling operations in existing HGPNNs are difficult to be interpreted. In this paper, we propose a new interpretable graph pooling framework - CommPOOL, that can capture and preserve the hierarchical community structure of graphs in the graph r…
GALUX inc.（GALUX公司）；Seoul National University（首尔国立大学）
In drug discovery, aqueous solubility is an important pharmacokinetic property which affects absorption and assay availability of drug. Thus, in silico prediction of solubility has been studied for its utility in virtual screening and lead optimization. Recently, machine learning (ML) methods using experimental data has been popular because physics-based methods like quantum mechanics and molecular dynamics are not suitable for high-throughput tasks due to its computational costs. However, ML method can exhibit over-fitting problem in a data-deficient condition, and this is the case for most c…
This work generalizes graph neural networks (GNNs) beyond those based on the Weisfeiler-Lehman (WL) algorithm, graph Laplacians, and diffusions. Our approach, denoted Relational Pooling (RP), draws from the theory of finite partial exchangeability to provide a framework with maximal representation power for graphs. RP can work with existing graph representation models and, somewhat counterintuitively, can make them even more powerful than the original WL isomorphism test. Additionally, RP allows architectures like Recurrent Neural Networks and Convolutional Neural Networks to be used in a theo…
NORCE, the Norwegian Research Centre（挪威研究中心NORCE）；Università della Svizzera italiana（意大利语区大学（瑞士））；Politecnico di Milano（米兰理工大学）
Spectral clustering (SC) is a popular clustering technique to find strongly connected communities on a graph. SC can be used in Graph Neural Networks (GNNs) to implement pooling operations that aggregate nodes belonging to the same cluster. However, the eigendecomposition of the Laplacian is expensive and, since clustering results are graph-specific, pooling methods based on SC must perform a new optimization for each new sample. In this paper, we propose a graph clustering approach that addresses these limitations of SC. We formulate a continuous relaxation of the normalized minCUT problem an…
Yale University（耶鲁大学）；Yale School of Medicine（耶鲁大学医学院）；Facebook AI Research（Facebook人工智能研究院）
Understanding how certain brain regions relate to a specific neurological disorder has been an important area of neuroimaging research. A promising approach to identify the salient regions is using Graph Neural Networks (GNNs), which can be used to analyze graph structured data, e.g. brain networks constructed by functional magnetic resonance imaging (fMRI). We propose an interpretable GNN framework with a novel salient region selection mechanism to determine neurological brain biomarkers associated with disorders. Specifically, we design novel regularized pooling layers that highlight salient…
Towards Interpretable Drug-Drug Interaction Prediction: A Graph-Based Approach with Molecular and Network-Level Explanations
Data Science Institute, School of Mathematics, Shandong University（数据科学研究院、数学学院、山东大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（数学与系统科学研究院、中国科学院）；Data Science Institute, Shandong University（数据科学研究院、山东大学）
Princeton University（普林斯顿大学）；University of Technology Sydney（悉尼科技大学）；Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克数学科学研究所）；University of New South Wales（新南威尔士大学）；Zhejiang Normal University（浙江师范大学）；University of Cambridge（剑桥大学）
Graph neural networks (GNNs) extends the functionality of traditional neural networks to graph-structured data. Similar to CNNs, an optimized design of graph convolution and pooling is key to success. Borrowing ideas from physics, we propose a path integral based graph neural networks (PAN) for classification and regression tasks on graphs. Specifically, we consider a convolution operation that involves every path linking the message sender and receiver with learnable weights depending on the path length, which corresponds to the maximal entropy random walk. It generalizes the graph Laplacian …
Advancing Graph Neural Networks with HL-HGAT: A Hodge-Laplacian and Attention Mechanism Approach for Heterogeneous Graph-Structured Data
National University of Singapore（新加坡国立大学）；the Hong Kong Polytechnic University（香港理工大学）；Fujian Agriculture and Forestry University（福建农林大学）；the University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Hierarchical Pooling and Explainability in Graph Neural Networks for Tumor and Tissue-of-Origin Classification Using RNA-seq Data
Institute of Informatics（信息学院）；Universidade Federal do Rio Grande do Sul（鲁汶联邦大学）；Bioinformatics Core（生物信息学核心）；Hospital de Clínicas de Porto Alegre（波多利亚临床医院）
Information Technologies Institute, Centre for Research and Technology Hellas（希腊研究与技术中心信息技术研究所）
In this paper, a novel multi-view methodology for graph-based neural networks is proposed. A systematic and methodological adaptation of the key concepts of classical deep learning methods such as convolution, pooling and multi-view architectures is developed for the context of non-Euclidean manifolds. The aim of the proposed work is to present a novel multi-view graph convolution layer, as well as a new view pooling layer making use of: a) a new hybrid Laplacian that is adjusted based on feature distance metric learning, b) multiple trainable representations of a feature matrix of a graph, us…
Convolutional layers within graph neural networks operate by aggregating information about local neighbourhood structures; one common way to encode such substructures is through random walks. The distribution of these random walks evolves according to a diffusion equation defined using the graph Laplacian. We extend this approach by leveraging classic mathematical results about hypo-elliptic diffusions. This results in a novel tensor-valued graph operator, which we call the hypo-elliptic graph Laplacian. We provide theoretical guarantees and efficient low-rank approximation algorithms. In part…
We propose a novel graph pooling operation using cliques as the unit pool. As this approach is purely topological, rather than featural, it is more readily interpretable, a better analogue to image coarsening than filtering or pruning techniques, and entirely nonparametric. The operation is implemented within graph convolution network (GCN) and GraphSAGE architectures and tested against standard graph classification benchmarks. In addition, we explore the backwards compatibility of the pooling to regular graphs, demonstrating competitive performance when replacing two-by-two pooling in standar…
We propose a combination of a variational autoencoder and a transformer based model which fully utilises graph convolutional and graph pooling layers to operate directly on graphs. The transformer model implements a novel node encoding layer, replacing the position encoding typically used in transformers, to create a transformer with no position information that operates on graphs, encoding adjacent node properties into the edge generation process. The proposed model builds on graph generative work operating on graphs with edge features, creating a model that offers improved scalability with t…
Department of Computing, Imperial College London, UK（伦敦帝国学院计算机系）；Department of Mathematics, Imperial College London, UK（伦敦帝国学院数学系）；London School of Geometry（几何与数论研究所）
Temple University（天普大学）；East China Normal University（华东师范大学）；Fudan University（复旦大学）；Georgia Institute of Technology（佐治亚理工学院）
Graph neural networks are promising architecture for learning and inference with graph-structured data. Yet difficulties in modelling the ``parts'' and their ``interactions'' still persist in terms of graph classification, where graph-level representations are usually obtained by squeezing the whole graph into a single vector through graph pooling. From complex systems point of view, mixing all the parts of a system together can affect both model interpretability and predictive performance, because properties of a complex system arise largely from the interaction among its components. We analy…
Sea-Net: A Small and Inherently Interpretable Neural Network for Time Series Classification
Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL)（里尔信息、信号与自动化研究中心）；University of Lille（里尔大学）；CNRS（法国国家科学研究中心）；Centrale Lille（里尔中央理工学院）
Shanghai Jiao Tong University（上海交通大学）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
Graph neural networks have attracted wide attentions to enable representation learning of graph data in recent works. In complement to graph convolution operators, graph pooling is crucial for extracting hierarchical representation of graph data. However, most recent graph pooling methods still fail to efficiently exploit the geometry of graph data. In this paper, we propose a novel graph pooling strategy that leverages node proximity to improve the hierarchical representation learning of graph data with their multi-hop topology. Node proximity is obtained by harmonizing the kernel representat…
Tiered graph autoencoders provide the architecture and mechanisms for learning tiered latent representations and latent spaces for molecular graphs that explicitly represent and utilize groups (e.g., functional groups). This enables the utilization and exploration of tiered molecular latent spaces, either individually - the node (atom) tier, the group tier, or the graph (molecule) tier - or jointly, as well as navigation across the tiers. In this paper, we discuss the use of tiered graph autoencoders together with graph prediction for molecular graphs. We show features of molecular graphs used…
Graduate School of Science and Technology（科技研究生院）；Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Graphs are a highly expressive data structure, but it is often difficult for humans to find patterns from a complex graph. Hence, generating human-interpretable sequences from graphs have gained interest, called graph2seq learning. It is expected that the compositionality in a graph can be associated to the compositionality in the output sequence in many graph2seq tasks. Therefore, applying compositionality-aware GNN architecture would improve the model performance. In this study, we adopt the multi-level attention pooling (MLAP) architecture, that can aggregate graph representations from mult…
FIGNN: Feature-Specific Interpretability for Graph Neural Network Surrogate Models
Department of Mechanical Engineering, Pennsylvania State University, University Park, 16802, PA, USA（宾夕法尼亚州立大学机械工程系）；College of Information Sciences and Technology, Pennsylvania State University, University Park, 16802, PA, USA（宾夕法尼亚州立大学信息科学与技术学院）；Power Systems Division, Argonne National Laboratory, Lemont, 60439, IL, USA（阿贡国家实验室电力系统分部）
Graph Convolutional Networks (GCNs) have shown to be effective in handling unordered data like point clouds and meshes. In this work we propose novel approaches for graph convolution, pooling and unpooling, inspired from finite differences and algebraic multigrid frameworks. We form a parameterized convolution kernel based on discretized differential operators, leveraging the graph mass, gradient and Laplacian. This way, the parameterization does not depend on the graph structure, only on the meaning of the network convolutions as differential operators. To allow hierarchical representations o…
University of Pittsburgh（匹兹堡大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Recently brain networks have been widely adopted to study brain dynamics, brain development and brain diseases. Graph representation learning techniques on brain functional networks can facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. However, current graph learning techniques have several issues on brain network mining. Firstly, most current graph learning models are designed for unsigned graph, which hinders the analysis of many signed network data (e.g., brain functional networks). Meanwhile, the insufficiency of brain network data limits …
In this paper, we consider the interpretability of the foundational Laplacian-based semi-supervised learning approaches on graphs. We introduce a novel flow-based learning framework that subsumes the foundational approaches and additionally provides a detailed, transparent, and easily understood expression of the learning process in terms of graph flows. As a result, one can visualize and interactively explore the precise subgraph along which the information from labeled nodes flows to an unlabeled node of interest. Surprisingly, the proposed framework avoids trading accuracy for interpretabil…
Unfolded Laplacian Spectral Embedding: A Theoretically Grounded Approach to Dynamic Network Representation
Department of Mathematical Informatics, Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院数学与信息学系）；Mathematics and Informatics Center, Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院数学与信息学中心）；The Canon Institute for Global Studies（Canon全球研究机构）
Yale Department of Computer Science（耶鲁大学计算机科学系）；Yale Department of Genetics（耶鲁大学遗传学系）；Université de Montréal, Dept. of Math. & Stat.（蒙特利尔大学数学与统计系）；Mila（米拉研究院）
While neural networks are powerful approximators used to classify or embed data into lower dimensional spaces, they are often regarded as black boxes with uninterpretable features. Here we propose Graph Spectral Regularization for making hidden layers more interpretable without significantly impacting performance on the primary task. Taking inspiration from spatial organization and localization of neuron activations in biological networks, we use a graph Laplacian penalty to structure the activations within a layer. This penalty encourages activations to be smooth either on a predetermined gra…
National University of Singapore（新加坡国立大学）；University of Wisconsin（威斯康星大学）；N.1 Institute for Health（N.1健康研究所）
This paper revisits spectral graph convolutional neural networks (graph-CNNs) given in Defferrard (2016) and develops the Laplace-Beltrami CNN (LB-CNN) by replacing the graph Laplacian with the LB operator. We then define spectral filters via the LB operator on a graph. We explore the feasibility of Chebyshev, Laguerre, and Hermite polynomials to approximate LB-based spectral filters and define an update of the LB operator for pooling in the LBCNN. We employ the brain image data from Alzheimer's Disease Neuroimaging Initiative (ADNI) and demonstrate the use of the proposed LB-CNN. Based on the…
Université de Rennes（雷恩大学）；ENSAI（国立统计与信息分析学校）；CNRS（法国国家科学研究中心）；CREST-UMR 9194；University of Southern California（南加州大学）
In this paper, we explore the topic of graph learning from the perspective of the Irregularity-Aware Graph Fourier Transform, with the goal of learning the graph signal space inner product to better model data. We propose a novel method to learn a graph with smaller edge weight upper bounds compared to combinatorial Laplacian approaches. Experimentally, our approach yields much sparser graphs compared to a combinatorial Laplacian approach, with a more interpretable model.
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；George Mason University（乔治梅森大学）；Rutgers University（罗格斯大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhengzhou University（郑州大学）
Unlike the traditional dock-based systems, dockless bike-sharing systems are more convenient for users in terms of flexibility. However, the flexibility of these dockless systems comes at the cost of management and operation complexity. Indeed, the imbalanced and dynamic use of bikes leads to mandatory rebalancing operations, which impose a critical need for effective bike traffic flow prediction. While efforts have been made in developing traffic flow prediction models, existing approaches lack interpretability, and thus have limited value in practical deployment. To this end, we propose an I…
GeoPro-Net: Learning Interpretable Spatiotemporal Prediction Models through Statistically-Guided Geo-Prototyping
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Georgia Institute of Technology（佐治亚理工学院）；University of Iowa（爱荷华大学）；Fudan University（复旦大学）；Logistics and Supply Chain MultiTech R&D Centre（物流与供应链多元技术研发中心）
Analytics of financial data is inherently a Big Data paradigm, as such data are collected over many assets, asset classes, countries, and time periods. This represents a challenge for modern machine learning models, as the number of model parameters needed to process such data grows exponentially with the data dimensions; an effect known as the Curse-of-Dimensionality. Recently, Tensor Decomposition (TD) techniques have shown promising results in reducing the computational costs associated with large-dimensional financial models while achieving comparable performance. However, tensor models ar…
Physically Interpretable Representation Learning with Gaussian Mixture Variational AutoEncoder (GM-VAE)
Institute for Computational and Mathematical Engineering, Stanford University（计算与数学工程研究所，斯坦福大学）；Department of Mechanical Engineering, Stanford University（机械工程系，斯坦福大学）；National Renewable Energy Laboratory（国家可再生能源实验室）；Ann and H.J. Smead Department of Aerospace Engineering Sciences, University of Colorado, Boulder（安与H.J.斯迈德航空航天工程科学系，科罗拉多大学，博尔德）
University of Science and Technology of China（中国科学技术大学）；Sea AI Lab（Sea人工智能实验室）；National University of Singapore（新加坡国立大学）
In graph classification, attention and pooling-based graph neural networks (GNNs) prevail to extract the critical features from the input graph and support the prediction. They mostly follow the paradigm of learning to attend, which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm makes GNN classifiers recklessly absorb all the statistical correlations between input features and labels in the training data, without distinguishing the causal and noncausal effects of features. Instead of underscoring the causal features, the attended …
Shell Information Technology Limited（Shell信息科技有限公司）；University College London（伦敦大学学院）；Department of Computer Science（计算机科学系）；Systemic Risk Centre（系统性风险中心）；London School of Economics and Political Sciences（伦敦政治经济学院）；Shell Information Technology International B.V.（Shell信息科技国际有限公司）
Geometric Data Analytics, Inc.（几何数据分析公司）；Duke University（杜克大学）
We propose a graph spectral representation of time series data that 1) is parsimoniously encoded to user-demanded resolution; 2) is unsupervised and performant in data-constrained scenarios; 3) captures event and event-transition structure within the time series; and 4) has near-linear computational complexity in both signal length and ambient dimension. This representation, which we call Laplacian Events Signal Segmentation (LESS), can be computed on time series of arbitrary dimension and originating from sensors of arbitrary type. Hence, time series originating from sensors of heterogeneous …
EEG-SeeGraph: Interpreting functional connectivity disruptions in dementias via sparse-explanatory dynamic EEG-graph learning
Harbin Institute of Technology, Shenzhen, China（哈尔滨工业大学（深圳））；Institute of Neurological Diseases, Shenzhen Bay Laboratory, Shenzhen, China（深圳湾实验室神经疾病研究院）
