We investigate the representation power of graph neural networks in the semi-supervised node classification task under heterophily or low homophily, i.e., in networks where connected nodes may have different class labels and dissimilar features. Many popular GNNs fail to generalize to this setting, and are even outperformed by models that ignore the graph structure (e.g., multilayer perceptrons). Motivated by this limitation, we identify a set of key designs -- ego- and neighbor-embedding separation, higher-order neighborhoods, and combination of intermediate representations -- that boost learning from the graph structure under heterophily. We combine them into a graph neural network, H2GCN, which we use as the base method to empirically evaluate the effectiveness of the identified designs. Going beyond the traditional benchmarks with strong homophily, our empirical analysis shows that the identified designs increase the accuracy of GNNs by up to 40% and 27% over models without them on synthetic and real networks with heterophily, respectively, and yield competitive performance under homophily.
Unsupervised graph representation learning aims to distill various graph information into a downstream task-agnostic dense vector embedding. However, existing graph representation learning approaches are designed mainly under the node homophily assumption: connected nodes tend to have similar labels and optimize performance on node-centric downstream tasks. Their design is apparently against the task-agnostic principle and generally suffers poor performance in tasks, e.g., edge classification, that demands feature signals beyond the node-view and homophily assumption. To condense different fea…
Graph Neural Networks (GNNs) show strong expressive power on graph data mining, by aggregating information from neighbors and using the integrated representation in the downstream tasks. The same aggregation methods and parameters for each node in a graph are used to enable the GNNs to utilize the homophily relational data. However, not all graphs are homophilic, even in the same graph, the distributions may vary significantly. Using the same convolution over all nodes may lead to the ignorance of various graph patterns. Furthermore, many existing GNNs integrate node features and structure ide…
What Is Missing In Homophily? Disentangling Graph Homophily For Graph Neural Networks
Centre for Info. Sciences and Systems（信息科学与系统中心）；Nanyang Technological University（南洋理工大学）；McGill University（麦吉尔大学）；Mila - Quebec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）
University of Michigan（密歇根大学）；Adobe Research（Adobe研究院）；Intel Labs（英特尔实验室）
Graph Neural Networks (GNNs) have proven to be useful for many different practical applications. However, many existing GNN models have implicitly assumed homophily among the nodes connected in the graph, and therefore have largely overlooked the important setting of heterophily, where most connected nodes are from different classes. In this work, we propose a novel framework called CPGNN that generalizes GNNs for graphs with either homophily or heterophily. The proposed framework incorporates an interpretable compatibility matrix for modeling the heterophily or homophily level in the graph, w…
Not All Nodes Are Created Equal: Homophily-Aware Stratification for Stable GNN Evaluation
Universität Bayreuth（拜罗伊特大学）；FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
The Australian National University（澳大利亚国立大学）；Pohang University of Science and Technology（浦项科技大学）
Graph neural networks (GNNs) have been extensively studied for prediction tasks on graphs. As pointed out by recent studies, most GNNs assume local homophily, i.e., strong similarities in local neighborhoods. This assumption however limits the generalizability power of GNNs. To address this limitation, we propose a flexible GNN model, which is capable of handling any graphs without being restricted by their underlying homophily. At its core, this model adopts a node attention mechanism based on multiple learnable spectral filters; therefore, the aggregation scheme is learned adaptively for eac…
Software Engineering Institute of Guangzhou（广州软件学院）；Guangdong University of Technology（广东工业大学）；Guangdong Polytechnic Normal University（广东技术师范大学）
Graph Neural Networks (GNNs) are a promising deep learning approach for circumventing many real-world problems on graph-structured data. However, these models usually have at least one of four fundamental limitations: over-smoothing, over-fitting, difficult to train, and strong homophily assumption. For example, Simple Graph Convolution (SGC) is known to suffer from the first and fourth limitations. To tackle these limitations, we identify a set of key designs including (D1) dilated convolution, (D2) multi-channel learning, (D3) self-attention score, and (D4) sign factor to boost learning from…
Unsupervised graph representation learning (UGRL) has drawn increasing research attention and achieved promising results in several graph analytic tasks. Relying on the homophily assumption, existing UGRL methods tend to smooth the learned node representations along all edges, ignoring the existence of heterophilic edges that connect nodes with distinct attributes. As a result, current methods are hard to generalize to heterophilic graphs where dissimilar nodes are widely connected, and also vulnerable to adversarial attacks. To address this issue, we propose a novel unsupervised Graph Represe…
Graph Echo State Networks (GESN) have already demonstrated their efficacy and efficiency in graph classification tasks. However, semi-supervised node classification brought out the problem of over-smoothing in end-to-end trained deep models, which causes a bias towards high homophily graphs. We evaluate for the first time GESN on node classification tasks with different degrees of homophily, analyzing also the impact of the reservoir radius. Our experiments show that reservoir models are able to achieve better or comparable accuracy with respect to fully trained deep models that implement ad h…
Department of Sociology, Cornell University（康奈尔大学社会学系）；Department of Sociology, Dartmouth College（达特茅斯学院社会学系）；Qatar Computing Research Institute（卡塔尔计算研究所）
In online social networks, it is common to use predictions of node categories to estimate measures of homophily and other relational properties. However, online social network data often lacks basic demographic information about the nodes. Researchers must rely on predicted node attributes to estimate measures of homophily, but little is known about the validity of these measures. We show that estimating homophily in a network can be viewed as a dyadic prediction problem, and that homophily estimates are unbiased when dyad-level residuals sum to zero in the network. Node-level prediction model…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Georgia Institute of Technology（佐治亚理工学院）；KAIST（韩国科学技术院）；Emory University（埃默里大学）
Graph Neural Networks (GNNs) are de facto node classification models in graph structured data. However, during testing-time, these algorithms assume no data shift, i.e., $\Pr_\text{train}(X,Y) = \Pr_\text{test}(X,Y)$. Domain adaption methods can be adopted for data shift, yet most of them are designed to only encourage similar feature distribution between source and target data. Conditional shift on classes can still affect such adaption. Fortunately, graph yields graph homophily across different data distributions. In response, we propose Shift-Robust Node Classification (SRNC) to address the…
SCNode: Spatial and Contextual Coordinates for Graph Representation Learning
Department of Mathematical Science（数学科学系）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；Department of Mathematics（数学系）；Florida State University（佛罗里达州立大学）；Department of Computer Science（计算机科学系）；AI Initiative（人工智能计划）；University of Central Florida（中央佛罗里达大学）
GRAIN: Multi-Granular and Implicit Information Aggregation Graph Neural Network for Heterophilous Graphs
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
Graph Neural Networks (GNNs) have shown excellent performance on graphs that exhibit strong homophily with respect to the node labels i.e. connected nodes have same labels. However, they perform poorly on heterophilic graphs. Recent approaches have typically modified aggregation schemes, designed adaptive graph filters, etc. to address this limitation. In spite of this, the performance on heterophilic graphs can still be poor. We propose a simple alternative method that exploits Truncated Singular Value Decomposition (TSVD) of topological structure and node features. Our approach achieves up t…
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
Graph Convolutional Networks (GCNs) have been widely applied in various fields due to their significant power on processing graph-structured data. Typical GCN and its variants work under a homophily assumption (i.e., nodes with same class are prone to connect to each other), while ignoring the heterophily which exists in many real-world networks (i.e., nodes with different classes tend to form edges). Existing methods deal with heterophily by mainly aggregating higher-order neighborhoods or combing the immediate representations, which leads to noise and irrelevant information in the result. Bu…
Graph Neural Networks (GNNs) achieve state-of-the-art performance on graph-structured data across numerous domains. Their underlying ability to represent nodes as summaries of their vicinities has proven effective for homophilous graphs in particular, in which same-type nodes tend to connect. On heterophilous graphs, in which different-type nodes are likely connected, GNNs perform less consistently, as neighborhood information might be less representative or even misleading. On the other hand, GNN performance is not inferior on all heterophilous graphs, and there is a lack of understanding of …
Towards Robust Graph Structural Learning Beyond Homophily via Preserving Neighbor Similarity
Dept. of Computer Science（计算机科学系）；Hong Kong Chu Hai College（香港 Chu Hai 学院）；Dept. of Computing（计算系）；The Hong Kong Polytechnic University（香港理工大学）；School of Cyber Science and Engineering（网络安全科学与工程学院）；Shanghai Jiao Tong University（上海交通大学）；Shanghai CESI Technology Co., Ltd（上海 CESI 技术有限公司）；China Electronics Standardization Institute（中国电子标准化研究院）
McGill University（麦吉尔大学）；Mila；Facebook AI Research（Facebook人工智能研究院）
Graph Neural Networks (GNNs) are deep learning methods which provide the current state of the art performance in node classification tasks. GNNs often assume homophily -- neighboring nodes having similar features and labels--, and therefore may not be at their full potential when dealing with non-homophilic graphs. In this work, we focus on addressing this limitation and enable Graph Attention Networks (GAT), a commonly used variant of GNNs, to explore the structural information within each graph locality. Inspired by the positional encoding in the Transformers, we propose a framework, termed …
Homophily modulates double descent generalization in graph convolution networks
Universität Basel（巴塞尔大学）；University of Science and Technology of China（中国科学技术大学）；Nanjing Normal University（南京师范大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Pennsylvania State University（宾夕法尼亚州立大学）
University of Mannheim（曼海姆大学）；RWTH Aachen University（亚琛工业大学）；GESIS - Leibniz Institute for the Social Sciences（GESIS-莱布尼茨社会科学研究所）；Complexity Science Hub Vienna（维也纳复杂性科学中心）
Limits of message passing for node classification: How class-bottlenecks restrict signal-to-noise ratio
Department of Mathematics（数学系）；Department of Computing（计算系）；UKRI Centre for Doctoral Training in AI for Healthcare（英国研究与创新人工智能医疗健康博士培训中心）；Imperial College London（帝国理工学院）；MIT Institute for Data, Systems, and Society（MIT数据、系统与社会研究所）；I-X Centre for AI in Science（I-X人工智能科学中心）；EPSRC Centre for the Mathematics of Precision Healthcare（工程与物理科学研究委员会精准医疗数学中心）
Many widely used datasets for graph machine learning tasks have generally been homophilous, where nodes with similar labels connect to each other. Recently, new Graph Neural Networks (GNNs) have been developed that move beyond the homophily regime; however, their evaluation has often been conducted on small graphs with limited application domains. We collect and introduce diverse non-homophilous datasets from a variety of application areas that have up to 384x more nodes and 1398x more edges than prior datasets. We further show that existing scalable graph learning and graph minibatching techn…
IBM Research（IBM研究院）；Indian Institute of Science（印度科学学院）
Network representation learning has traditionally been used to find lower dimensional vector representations of the nodes in a network. However, there are very important edge driven mining tasks of interest to the classical network analysis community, which have mostly been unexplored in the network embedding space. For applications such as link prediction in homogeneous networks, vector representation (i.e., embedding) of an edge is derived heuristically just by using simple aggregations of the embeddings of the end vertices of the edge. Clearly, this method of deriving edge embedding is subo…
