Many recent works have studied the performance of Graph Neural Networks (GNNs) in the context of graph homophily - a label-dependent measure of connectivity. Traditional GNNs generate node embeddings by aggregating information from a node's neighbors in the graph. Recent results in node classification tasks show that this local aggregation approach performs poorly in graphs with low homophily (heterophilic graphs). Several mechanisms have been proposed to improve the accuracy of GNNs on such graphs by increasing the aggregation range of a GNN layer, either through multi-hop aggregation, or through long-range aggregation from distant nodes. In this paper, we show that properly tuned classical GNNs and multi-layer perceptrons match or exceed the accuracy of recent long-range aggregation methods on heterophilic graphs. Thus, our results highlight the need for alternative datasets to benchmark long-range GNN aggregation mechanisms. We also show that homophily is a poor measure of the information in a node's local neighborhood and propose the Neighborhood Information Content(NIC) metric, which is a novel information-theoretic graph metric. We argue that NIC is more relevant for local aggregation methods as used by GNNs. We show that, empirically, it correlates better with GNN accuracy in node classification tasks than homophily.
Graph Neural Networks (GNNs) have shown excellent performance on graphs that exhibit strong homophily with respect to the node labels i.e. connected nodes have same labels. However, they perform poorly on heterophilic graphs. Recent approaches have typically modified aggregation schemes, designed adaptive graph filters, etc. to address this limitation. In spite of this, the performance on heterophilic graphs can still be poor. We propose a simple alternative method that exploits Truncated Singular Value Decomposition (TSVD) of topological structure and node features. Our approach achieves up t…
The Heterophilic Snowflake Hypothesis: Training and Empowering GNNs for Heterophilic Graphs
University of Science and Technology of China (USTC)（中国科学技术大学）；Tongji University（同济大学）；University of Minnesota（明尼苏达大学）；Tsinghua University（清华大学）；Oxford University（牛津大学）；Griffith University（格里菲斯大学）；RIKEN AIP（理化学研究所人工智能项目）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Language Models as Messengers: Enhancing Message Passing in Heterophilic Graph Learning
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；School of Information and Intelligent Science, Donghua University（东华大学信息与智能科学学院）
Unsupervised Graph Anomaly Detection via Multi-Hypersphere Heterophilic Graph Learning
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Shanghai University（上海大学）
Non-Dissipative Graph Propagation for Non-Local Community Detection
School of Engineering Mathematics and Technology（工程数学与技术学院）；Technology University of Bristol Bristol, UK（布里斯托尔技术大学（英国））；Dept. of Computer Science University of Pisa Pisa, Italy（计算机科学系，帕尔米斯大学（意大利））
All Nodes are created Not Equal: Node-Specific Layer Aggregation and Filtration for GNN
University of Science and Technology of China（中国科学技术大学）；Tongji University（同济大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Oxford（牛津大学）；Griffith University（格里菲斯大学）
The Australian National University（澳大利亚国立大学）；Pohang University of Science and Technology（浦项科技大学）
Graph neural networks (GNNs) have been extensively studied for prediction tasks on graphs. As pointed out by recent studies, most GNNs assume local homophily, i.e., strong similarities in local neighborhoods. This assumption however limits the generalizability power of GNNs. To address this limitation, we propose a flexible GNN model, which is capable of handling any graphs without being restricted by their underlying homophily. At its core, this model adopts a node attention mechanism based on multiple learnable spectral filters; therefore, the aggregation scheme is learned adaptively for eac…
Graph neural networks (GNNs) have demonstrated superior performance for semi-supervised node classification on graphs, as a result of their ability to exploit node features and topological information simultaneously. However, most GNNs implicitly assume that the labels of nodes and their neighbors in a graph are the same or consistent, which does not hold in heterophilic graphs, where the labels of linked nodes are likely to differ. Hence, when the topology is non-informative for label prediction, ordinary GNNs may work significantly worse than simply applying multi-layer perceptrons (MLPs) on…
Adapting to Heterophilic Graph Data with Structure-Guided Neighbor Discovery
Department of Signal Theory and Communications, King Juan Carlos University（信号理论与通信系，国王胡安·卡洛斯大学）；Department of Electrical and Computer Engineering, Rice University（电气与计算机工程系，里奇大学）
Heterophily-Agnostic Hypergraph Neural Networks with Riemannian Local Exchanger
Beijing University of Posts and Telecommunications（北京邮电大学）；North China Electric Power University（华北电力大学）；University of Kent（肯特大学）；Beihang University（北航）；University of Illinois（伊利诺伊大学）
Huazhong University of Science and Technology（华中科技大学）
The decoupled Graph Convolutional Network (GCN), a recent development of GCN that decouples the neighborhood aggregation and feature transformation in each convolutional layer, has shown promising performance for graph representation learning. Existing decoupled GCNs first utilize a simple neural network (e.g., MLP) to learn the hidden features of the nodes, then propagate the learned features on the graph with fixed steps to aggregate the information of multi-hop neighborhoods. Despite effectiveness, the aggregation operation, which requires the whole adjacency matrix as the input, is involve…
Graph Neural Networks (GNNs) have shown advantages in various graph-based applications. Most existing GNNs assume strong homophily of graph structure and apply permutation-invariant local aggregation of neighbors to learn a representation for each node. However, they fail to generalize to heterophilic graphs, where most neighboring nodes have different labels or features, and the relevant nodes are distant. Few recent studies attempt to address this problem by combining multiple hops of hidden representations of central nodes (i.e., multi-hop-based approaches) or sorting the neighboring nodes …
SCNode: Spatial and Contextual Coordinates for Graph Representation Learning
Department of Mathematical Science（数学科学系）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；Department of Mathematics（数学系）；Florida State University（佛罗里达州立大学）；Department of Computer Science（计算机科学系）；AI Initiative（人工智能计划）；University of Central Florida（中央佛罗里达大学）
Addressing Graph Heterogeneity and Heterophily from A Spectral Perspective
Beijing University of Posts and Telecommunications（北京邮电大学）；National University of Singapore（新加坡国立大学）；Beihang University（北京航空航天大学）；Hangzhou Dianzi University（杭州电子科技大学）
Remedying Coarsening-Based GNN Training under Heterophily via Adaptive Complementary Enhancement
Hong Kong Baptist University（香港浸会大学）；Renmin University of China（中国人民大学）；Chinese Academy of Sciences（中国科学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Sun Yat-sen University（中山大学）
Graph Neural Networks (GNNs) extend basic Neural Networks (NNs) by using graph structures based on the relational inductive bias (homophily assumption). While GNNs have been commonly believed to outperform NNs in real-world tasks, recent work has identified a non-trivial set of datasets where their performance compared to NNs is not satisfactory. Heterophily has been considered the main cause of this empirical observation and numerous works have been put forward to address it. In this paper, we first revisit the widely used homophily metrics and point out that their consideration of only graph…
Graph Neural Networks (GNNs) learn low dimensional representations of nodes by aggregating information from their neighborhood in graphs. However, traditional GNNs suffer from two fundamental shortcomings due to their local ($l$-hop neighborhood) aggregation scheme. First, not all nodes in the neighborhood carry relevant information for the target node. Since GNNs do not exclude noisy nodes in their neighborhood, irrelevant information gets aggregated, which reduces the quality of the representation. Second, traditional GNNs also fail to capture long-range non-local dependencies between nodes.…
Due to the over-smoothing issue, most existing graph neural networks can only capture limited dependencies with their inherently finite aggregation layers. To overcome this limitation, we propose a new kind of graph convolution, called Graph Implicit Nonlinear Diffusion (GIND), which implicitly has access to infinite hops of neighbors while adaptively aggregating features with nonlinear diffusion to prevent over-smoothing. Notably, we show that the learned representation can be formalized as the minimizer of an explicit convex optimization objective. With this property, we can theoretically ch…
Beihang University（北京航空航天大学）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Shen Yuan Honors College, Beihang University（北京航空航天大学沈元荣誉学院）；Hebei University of Technology（河北工业大学）；School of Artificial Intelligence, Hebei University of Technology（河北工业大学人工智能学院）
Full-Spectrum Graph Neural Networks: Expressive and Scalable
Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore 637371, Singapore（数学科学学院，物理与数学科学学院，南洋理工大学，新加坡637371，新加坡）
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；School of Cyberspace, Hangzhou Dianzi University（杭州电子科技大学网络空间安全学院）
Graph Convolutional Network (GCN) has shown remarkable potential of exploring graph representation. However, the GCN aggregating mechanism fails to generalize to networks with heterophily where most nodes have neighbors from different classes, which commonly exists in real-world networks. In order to make the propagation and aggregation mechanism of GCN suitable for both homophily and heterophily (or even their mixture), we introduce block modeling into the framework of GCN so that it can realize "block-guided classified aggregation", and automatically learn the corresponding aggregation rules…
In contrast to image/text data whose order can be used to perform non-local feature aggregation in a straightforward way using the pooling layers, graphs lack the tensor representation and mostly the element-wise max/mean function is utilized to aggregate the locally extracted feature vectors. In this paper, we present a novel approach for global feature aggregation in Graph Neural Networks (GNNs) which utilizes a Latent Fixed Data Structure (LFDS) to aggregate the extracted feature vectors. The locally extracted feature vectors are sorted/distributed on the LFDS and a latent neural network (C…
Modern graph neural networks (GNNs) learn node embeddings through multilayer local aggregation and achieve great success in applications on assortative graphs. However, tasks on disassortative graphs usually require non-local aggregation. In addition, we find that local aggregation is even harmful for some disassortative graphs. In this work, we propose a simple yet effective non-local aggregation framework with an efficient attention-guided sorting for GNNs. Based on it, we develop various non-local GNNs. We perform thorough experiments to analyze disassortative graph datasets and evaluate ou…
Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph
National University of Singapore（新加坡国立大学）；University of Washington（华盛顿大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
SAHG: Sector-Anisotropic Hyperbolic Graph Model for Social Bot Detection
University of Leeds（利兹大学）；University of Science and Technology of China（中国科学技术大学）；South China Normal University（华南师范大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；Harbin University of Commerce（哈尔滨商业大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Peking University（北京大学）；University of California, Berkeley（加州大学伯克利分校）
We propose a dynamic neighborhood aggregation (DNA) procedure guided by (multi-head) attention for representation learning on graphs. In contrast to current graph neural networks which follow a simple neighborhood aggregation scheme, our DNA procedure allows for a selective and node-adaptive aggregation of neighboring embeddings of potentially differing locality. In order to avoid overfitting, we propose to control the channel-wise connections between input and output by making use of grouped linear projections. In a number of transductive node-classification experiments, we demonstrate the ef…
Institute for Digital Technologies, Loughborough University（拉夫堡大学数字技术研究所）
Federated learning has gained popularity as a solution to data availability and privacy challenges in machine learning. However, the aggregation process of local model updates to obtain a global model in federated learning is susceptible to malicious attacks, such as backdoor poisoning, label-flipping, and membership inference. Malicious users aim to sabotage the collaborative learning process by training the local model with malicious data. In this paper, we propose a novel robust aggregation approach based on recursive Euclidean distance calculation. Our approach measures the distance of the…
