Cellular sheaves equip graphs with a "geometrical" structure by assigning vector spaces and linear maps to nodes and edges. Graph Neural Networks (GNNs) implicitly assume a graph with a trivial underlying sheaf. This choice is reflected in the structure of the graph Laplacian operator, the properties of the associated diffusion equation, and the characteristics of the convolutional models that discretise this equation. In this paper, we use cellular sheaf theory to show that the underlying geometry of the graph is deeply linked with the performance of GNNs in heterophilic settings and their oversmoothing behaviour. By considering a hierarchy of increasingly general sheaves, we study how the ability of the sheaf diffusion process to achieve linear separation of the classes in the infinite time limit expands. At the same time, we prove that when the sheaf is non-trivial, discretised parametric diffusion processes have greater control than GNNs over their asymptotic behaviour. On the practical side, we study how sheaves can be learned from data. The resulting sheaf diffusion models have many desirable properties that address the limitations of classical graph diffusion equations (and corresponding GNN models) and obtain competitive results in heterophilic settings. Overall, our work provides new connections between GNNs and algebraic topology and would be of interest to both fields.
Oversmoothing as Representation Degeneracy in Neural Sheaf Diffusion
IUF – Leibniz Research Institute for Environmental Medicine（莱比锡环境医学研究所）；DNTOX GmbH（DNTOX公司）；Bioinformatics Group, Ruhr University Bochum（博德姆鲁尔大学生物信息学小组）；Swiss Centre for Applied Human Toxicology (SCAHT)（瑞士应用人类毒理学中心（SCAHT））
University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；Google Research（谷歌研究院）；University of California, Berkeley（加州大学伯克利分校）
In node classification tasks, graph convolutional neural networks (GCNs) have demonstrated competitive performance over traditional methods on diverse graph data. However, it is known that the performance of GCNs degrades with increasing number of layers (oversmoothing problem) and recent studies have also shown that GCNs may perform worse in heterophilous graphs, where neighboring nodes tend to belong to different classes (heterophily problem). These two problems are usually viewed as unrelated, and thus are studied independently, often at the graph filter level from a spectral perspective. W…
Seoul National University（首尔国立大学）；University of Michigan（密歇根大学）；Amazon Web Services（亚马逊云科技）
Heterogeneous graph neural networks (GNNs) achieve strong performance on node classification tasks in a semi-supervised learning setting. However, as in the simpler homogeneous GNN case, message-passing-based heterogeneous GNNs may struggle to balance between resisting the oversmoothing that may occur in deep models, and capturing long-range dependencies of graph structured data. Moreover, the complexity of this trade-off is compounded in the heterogeneous graph case due to the disparate heterophily relationships between nodes of different types. To address these issues, we propose a novel het…
Fudan University（复旦大学）；Shanghai Jiao Tong University（上海交通大学）；Amazon（亚马逊）；Renmin University of China（中国人民大学）
Despite the recent success of graph neural networks (GNN), common architectures often exhibit significant limitations, including sensitivity to oversmoothing, long-range dependencies, and spurious edges, e.g., as can occur as a result of graph heterophily or adversarial attacks. To at least partially address these issues within a simple transparent framework, we consider a new family of GNN layers designed to mimic and integrate the update rules of two classical iterative algorithms, namely, proximal gradient descent and iterative reweighted least squares (IRLS). The former defines an extensib…
University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；Twitter（推特）；DeepMind（深度思维）
A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that operates on a sheaf, an object that equips a graph with vector spaces over its nodes and edges and linear maps between these spaces. SNNs have been shown to have useful theoretical properties that help tackle issues arising from heterophily and over-smoothing. One complication intrinsic to these models is finding a good sheaf for the task to be solved. Previous works proposed two diametrically opposed approaches: manually constructing the sheaf based on domain knowledge and learning the sheaf end-to-end using gradient-ba…
Dynamic Sheaf Diffusion Networks with Adaptive Local Structure for Heterogeneous Spatio-Temporal Graph Learning
Peter L. Reichertz Institute for Medical Informatics（汉诺威医学院彼得·L·里赫茨医学信息学研究所）；Hannover Medical School（汉诺威医学院）；Lower Saxony Center for Artificial Intelligence and Causal Methods in Medicine (CAIMed)（下萨克森人工智能与医学因果方法中心（CAIMed））
ChronoSheaf: Joint Spatio-Temporal Spectral Filtering on Cellular Sheaves
Peter L. Reichertz Institute for Medical Informatics（彼得·L·赖歇茨医学信息学研究所）；Hannover Medical School（汉诺威医学院）；Lower Saxony Center for Artificial Intelligence and Causal Methods in Medicine (CAIMed)（下萨克森州人工智能与医学因果方法中心（CAIMed））
Demystifying Oversmoothing in Sheaf Neural Networks: An Index-Theoretic Criterion
Chern Institute of Mathematics（陈省身数学研究所）；Nankai University（南开大学）；School of Physical and Mathematical Sciences（物理与数学科学学院）；Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）
Gauge-Equivariant Graph Networks via Self-Interference Cancellation
Department of Artificial Intelligence, Sookmyung Women's University, Seoul, South Korea（首尔大学女子大学人工智能系）；Korea Advanced Institute of Science and Technology, Seoul, South Korea（韩国科学技术院）
Polynomial Neural Sheaf Diffusion: A Spectral Filtering Approach on Cellular Sheaves
Department of Computer Science and Technology, University of Cambridge（计算机科学与技术系，剑桥大学）；Department of Computer, Control and Management Engineering, Sapienza University（计算机、控制与管理工程系，萨皮恩扎大学）
Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework
Vinci4D；University of San Francisco（旧金山大学）；Technical University of Munich（慕尼黑技术大学）；Stanford University（斯坦福大学）；PolyShape；Imperial College London（伦敦帝国学院）；MCML
