We present a generalization of graph convolutional networks by generalizing the diffusion operation underlying this class of graph neural networks. These sheaf neural networks are based on the sheaf Laplacian, a generalization of the graph Laplacian that encodes additional relational structure parameterized by the underlying graph. The sheaf Laplacian and associated matrices provide an extended version of the diffusion operation in graph convolutional networks, providing a proper generalization for domains where relations between nodes are non-constant, asymmetric, and varying in dimension. We show that the resulting sheaf neural networks can outperform graph convolutional networks in domains where relations between nodes are asymmetric and signed.
University of Cambridge（剑桥大学）；Twitter（推特）；University of Oxford（牛津大学）
Cellular sheaves equip graphs with a "geometrical" structure by assigning vector spaces and linear maps to nodes and edges. Graph Neural Networks (GNNs) implicitly assume a graph with a trivial underlying sheaf. This choice is reflected in the structure of the graph Laplacian operator, the properties of the associated diffusion equation, and the characteristics of the convolutional models that discretise this equation. In this paper, we use cellular sheaf theory to show that the underlying geometry of the graph is deeply linked with the performance of GNNs in heterophilic settings and their ov…
University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；Twitter（推特）；DeepMind（深度思维）
A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that operates on a sheaf, an object that equips a graph with vector spaces over its nodes and edges and linear maps between these spaces. SNNs have been shown to have useful theoretical properties that help tackle issues arising from heterophily and over-smoothing. One complication intrinsic to these models is finding a good sheaf for the task to be solved. Previous works proposed two diametrically opposed approaches: manually constructing the sheaf based on domain knowledge and learning the sheaf end-to-end using gradient-ba…
Oversmoothing as Representation Degeneracy in Neural Sheaf Diffusion
IUF – Leibniz Research Institute for Environmental Medicine（莱比锡环境医学研究所）；DNTOX GmbH（DNTOX公司）；Bioinformatics Group, Ruhr University Bochum（博德姆鲁尔大学生物信息学小组）；Swiss Centre for Applied Human Toxicology (SCAHT)（瑞士应用人类毒理学中心（SCAHT））
Department of Computer Science and Technology, University of Cambridge（计算机科学与技术系，剑桥大学）；Department of Electrical and Computer Engineering, Northeastern University（电气与计算机工程系，东北大学）；Department of Computer, Control and Management Engineering, Sapienza University of Rome（计算机、控制与管理工程系，罗马萨皮恩扎大学）
Dynamic Sheaf Diffusion Networks with Adaptive Local Structure for Heterogeneous Spatio-Temporal Graph Learning
Peter L. Reichertz Institute for Medical Informatics（汉诺威医学院彼得·L·里赫茨医学信息学研究所）；Hannover Medical School（汉诺威医学院）；Lower Saxony Center for Artificial Intelligence and Causal Methods in Medicine (CAIMed)（下萨克森人工智能与医学因果方法中心（CAIMed））
ChronoSheaf: Joint Spatio-Temporal Spectral Filtering on Cellular Sheaves
Peter L. Reichertz Institute for Medical Informatics（彼得·L·赖歇茨医学信息学研究所）；Hannover Medical School（汉诺威医学院）；Lower Saxony Center for Artificial Intelligence and Causal Methods in Medicine (CAIMed)（下萨克森州人工智能与医学因果方法中心（CAIMed））
Graph convolutional networks are a popular class of deep neural network algorithms which have shown success in a number of relational learning tasks. Despite their success, graph convolutional networks exhibit a number of peculiar features, including a bias towards learning oversmoothed and homophilic functions, which are not easily diagnosed due to the complex nature of these algorithms. We propose to bridge this gap in understanding by studying the neural tangent kernel of sheaf convolutional networks--a topological generalization of graph convolutional networks. To this end, we derive a par…
Hypergraph Neural Sheaf Diffusion: A Symmetric Simplicial Set Framework for Higher-Order Learning
POSTECH；Center for Geometry and Physics, Institute for Basic Science (IBS)（几何与物理中心，基础科学研究所）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Polynomial Neural Sheaf Diffusion: A Spectral Filtering Approach on Cellular Sheaves
Department of Computer Science and Technology, University of Cambridge（计算机科学与技术系，剑桥大学）；Department of Computer, Control and Management Engineering, Sapienza University（计算机、控制与管理工程系，萨皮恩扎大学）
Demystifying Oversmoothing in Sheaf Neural Networks: An Index-Theoretic Criterion
Chern Institute of Mathematics（陈省身数学研究所）；Nankai University（南开大学）；School of Physical and Mathematical Sciences（物理与数学科学学院）；Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）
Queen Mary University of London（伦敦玛丽女王大学）；American University of Beirut（贝鲁特美国大学）
Graph Neural Networks have gained huge interest in the past few years. These powerful algorithms expanded deep learning models to non-Euclidean space and were able to achieve state of art performance in various applications including recommender systems and social networks. However, this performance is based on static graph structures assumption which limits the Graph Neural Networks performance when the data varies with time. Spatiotemporal Graph Neural Networks are extension of Graph Neural Networks that takes the time factor into account. Recently, various Spatiotemporal Graph Neural Networ…
In general, Graph Neural Networks(GNN) have been using a message passing method to aggregate and summarize information about neighbors to express their information. Nonetheless, previous studies have shown that the performance of graph neural networks becomes vulnerable when there are abnormal nodes in the neighborhood due to this message passing method. In this paper, inspired by the Neural Architecture Search method, we present an algorithm that recognizes abnormal nodes and automatically excludes them from information aggregation. Experiments on various real worlds datasets show that our pr…
Formal verification of neural networks is essential for their deployment in safety-critical areas. Many available formal verification methods have been shown to be instances of a unified Branch and Bound (BaB) formulation. We propose a novel framework for designing an effective branching strategy for BaB. Specifically, we learn a graph neural network (GNN) to imitate the strong branching heuristic behaviour. Our framework differs from previous methods for learning to branch in two main aspects. Firstly, our framework directly treats the neural network we want to verify as a graph input for the…
National University of Singapore（新加坡国立大学）；Nanjing University（南京大学）
Most of the successful deep neural network architectures are structured, often consisting of elements like convolutional neural networks and gated recurrent neural networks. Recently, graph neural networks have been successfully applied to graph structured data such as point cloud and molecular data. These networks often only consider pairwise dependencies, as they operate on a graph structure. We generalize the graph neural network into a factor graph neural network (FGNN) in order to capture higher order dependencies. We show that FGNN is able to represent Max-Product Belief Propagation, an …
David S. Hippocampus Department of Computer Science Cranberry-Lemon University（David S. Hippocampus 计算机科学系 Cranberry-Lemon 大学）；UiT The Arctic University of Norway（UiT 北极大学）；Norwegian Computing Center（挪威计算中心）；University of South Brittany（南布列塔尼大学）；University of Copenhagen（哥本哈根大学）
Logit-Boundary Geometric Belief Interfaces and Sparse Sheaf-Enclave Protocols: A Self-Contained Substrate for Secure Network Electronic Health Record (EHR) Interoperability
University of Pennsylvania（宾夕法尼亚大学）；Sapienza University of Rome（罗马第一大学）；Duke University（杜克大学）
In this work we introduce a convolution operation over the tangent bundle of Riemannian manifolds exploiting the Connection Laplacian operator. We use the convolution to define tangent bundle filters and tangent bundle neural networks (TNNs), novel continuous architectures operating on tangent bundle signals, i.e. vector fields over manifolds. We discretize TNNs both in space and time domains, showing that their discrete counterpart is a principled variant of the recently introduced Sheaf Neural Networks. We formally prove that this discrete architecture converges to the underlying continuous …
