Towards Robust Graph Structural Learning Beyond Homophily via Preserving Neighbor Similarity
Dept. of Computer Science（计算机科学系）；Hong Kong Chu Hai College（香港 Chu Hai 学院）；Dept. of Computing（计算系）；The Hong Kong Polytechnic University（香港理工大学）；School of Cyber Science and Engineering（网络安全科学与工程学院）；Shanghai Jiao Tong University（上海交通大学）；Shanghai CESI Technology Co., Ltd（上海 CESI 技术有限公司）；China Electronics Standardization Institute（中国电子标准化研究院）
Despite the tremendous success of graph-based learning systems in handling structural data, it has been widely investigated that they are fragile to adversarial attacks on homophilic graph data, where adversaries maliciously modify the semantic and topology information of the raw graph data to degrade the predictive performances. Motivated by this, a series of robust models are crafted to enhance the adversarial robustness of graph-based learning systems on homophilic graphs. However, the security of graph-based learning systems on heterophilic graphs remains a mystery to us. To bridge this gap, in this paper, we start to explore the vulnerability of graph-based learning systems regardless of the homophily degree, and theoretically prove that the update of the negative classification loss is negatively correlated with the pairwise similarities based on the powered aggregated neighbor features. The theoretical finding inspires us to craft a novel robust graph structural learning strategy that serves as a useful graph mining module in a robust model that incorporates a dual-kNN graph constructions pipeline to supervise the neighbor-similarity-preserved propagation, where the graph convolutional layer adaptively smooths or discriminates the features of node pairs according to their affluent local structures. In this way, the proposed methods can mine the ``better" topology of the raw graph data under diverse graph homophily and achieve more reliable data management on homophilic and heterophilic graphs.
University of Michigan（密歇根大学）；Lawrence Livermore National Lab（劳伦斯利弗莫尔国家实验室）；RWTH Aachen University（亚琛工业大学）
We study the task of node classification for graph neural networks (GNNs) and establish a connection between group fairness, as measured by statistical parity and equal opportunity, and local assortativity, i.e., the tendency of linked nodes to have similar attributes. Such assortativity is often induced by homophily, the tendency for nodes of similar properties to connect. Homophily can be common in social networks where systemic factors have forced individuals into communities which share a sensitive attribute. Through synthetic graphs, we study the interplay between locally occurring homoph…
East China Normal University（华东师范大学）；Microsoft Research Asia（微软亚洲研究院）；Nanyang Technological University（南洋理工大学）
We investigate graph neural networks on graphs with heterophily. Some existing methods amplify a node's neighborhood with multi-hop neighbors to include more nodes with homophily. However, it is a significant challenge to set personalized neighborhood sizes for different nodes. Further, for other homophilous nodes excluded in the neighborhood, they are ignored for information aggregation. To address these problems, we propose two models GloGNN and GloGNN++, which generate a node's embedding by aggregating information from global nodes in the graph. In each layer, both models learn a coefficien…
Tianjin University（天津大学）；Meituan（美团）；The Hong Kong Polytechnic University（香港理工大学）
Graph-Convolution-based methods have been successfully applied to representation learning on homophily graphs where nodes with the same label or similar attributes tend to connect with one another. Due to the homophily assumption of Graph Convolutional Networks (GCNs) that these methods use, they are not suitable for heterophily graphs where nodes with different labels or dissimilar attributes tend to be adjacent. Several methods have attempted to address this heterophily problem, but they do not change the fundamental aggregation mechanism of GCNs because they rely on summation operators to a…
University of Michigan（密歇根大学）；Carnegie Mellon University（卡内基梅隆大学）
We investigate the representation power of graph neural networks in the semi-supervised node classification task under heterophily or low homophily, i.e., in networks where connected nodes may have different class labels and dissimilar features. Many popular GNNs fail to generalize to this setting, and are even outperformed by models that ignore the graph structure (e.g., multilayer perceptrons). Motivated by this limitation, we identify a set of key designs -- ego- and neighbor-embedding separation, higher-order neighborhoods, and combination of intermediate representations -- that boost lear…
What Is Missing In Homophily? Disentangling Graph Homophily For Graph Neural Networks
Centre for Info. Sciences and Systems（信息科学与系统中心）；Nanyang Technological University（南洋理工大学）；McGill University（麦吉尔大学）；Mila - Quebec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）
Making Classic GNNs Strong Baselines Across Varying Homophily: A Smoothness-Generalization Perspective
Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems, Zhejiang University（浙江可及感知与智能系统重点实验室，浙江大学）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Software Technology, Zhejiang University（浙江大学软件学院）；China University of Mining and Technology（中国矿业大学）；Department of Computer Science, New York University Shanghai（纽约大学上海分校计算机科学系）
Rethinking Graph Contrastive Learning through Relative Similarity Preservation
Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences（中国科学院大学）；Hangzhou Institute for Advanced Study, UCAS（国科大杭州高等研究院）；School of Computing and Information Technology, Great Bay University（大湾区大学计算与信息技术学院）；Department of Computer and Information Science, IOTSC, University of Macau（澳门大学计算机与信息科学系）
Many recent works have studied the performance of Graph Neural Networks (GNNs) in the context of graph homophily - a label-dependent measure of connectivity. Traditional GNNs generate node embeddings by aggregating information from a node's neighbors in the graph. Recent results in node classification tasks show that this local aggregation approach performs poorly in graphs with low homophily (heterophilic graphs). Several mechanisms have been proposed to improve the accuracy of GNNs on such graphs by increasing the aggregation range of a GNN layer, either through multi-hop aggregation, or thr…
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
Graph Convolutional Networks (GCNs) have been widely applied in various fields due to their significant power on processing graph-structured data. Typical GCN and its variants work under a homophily assumption (i.e., nodes with same class are prone to connect to each other), while ignoring the heterophily which exists in many real-world networks (i.e., nodes with different classes tend to form edges). Existing methods deal with heterophily by mainly aggregating higher-order neighborhoods or combing the immediate representations, which leads to noise and irrelevant information in the result. Bu…
The crime forecasting is an important problem as it greatly contributes to urban safety. Typically, the goal of the problem is to predict different types of crimes for each geographical region (like a neighborhood or censor tract) in the near future. Since nearby regions usually have similar socioeconomic characteristics which indicate similar crime patterns, recent state-of-the-art solutions constructed a distance-based region graph and utilized Graph Neural Network (GNN) techniques for crime forecasting, because the GNN techniques could effectively exploit the latent relationships between ne…
University of Oregon Eugene OR USA；Pacific Northwest National Laboratory Richland WA USA；Adobe Research San Jose CA USA；Texas A\&M University College Station TX USA；Rensselaer Polytechnic Institute Troy NY USA；University of Oregon；Pacific Northwest National Laboratory；Adobe Research；Texas A\&M University；Rensselaer Polytechnic Institute
The Chinese University of Hong Kong（香港中文大学）；Hong Kong Baptist University（香港浸会大学）；The University of Sydney（悉尼大学）
Recently Graph Injection Attack (GIA) emerges as a practical attack scenario on Graph Neural Networks (GNNs), where the adversary can merely inject few malicious nodes instead of modifying existing nodes or edges, i.e., Graph Modification Attack (GMA). Although GIA has achieved promising results, little is known about why it is successful and whether there is any pitfall behind the success. To understand the power of GIA, we compare it with GMA and find that GIA can be provably more harmful than GMA due to its relatively high flexibility. However, the high flexibility will also lead to great d…
Microsoft Research Asia（微软亚洲研究院）；Beijing University of Technology（北京工业大学）；Peking University（北京大学）；Northeastern University（东北大学）
Graph Neural Networks (GNNs) are widely used on a variety of graph-based machine learning tasks. For node-level tasks, GNNs have strong power to model the homophily property of graphs (i.e., connected nodes are more similar) while their ability to capture the heterophily property is often doubtful. This is partially caused by the design of the feature transformation with the same kernel for the nodes in the same hop and the followed aggregation operator. One kernel cannot model the similarity and the dissimilarity (i.e., the positive and negative correlation) between node features simultaneous…
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；School of Cyberspace, Hangzhou Dianzi University（杭州电子科技大学网络空间安全学院）
Graph Convolutional Network (GCN) has shown remarkable potential of exploring graph representation. However, the GCN aggregating mechanism fails to generalize to networks with heterophily where most nodes have neighbors from different classes, which commonly exists in real-world networks. In order to make the propagation and aggregation mechanism of GCN suitable for both homophily and heterophily (or even their mixture), we introduce block modeling into the framework of GCN so that it can realize "block-guided classified aggregation", and automatically learn the corresponding aggregation rules…
Graph Neural Networks (GNNs) show strong expressive power on graph data mining, by aggregating information from neighbors and using the integrated representation in the downstream tasks. The same aggregation methods and parameters for each node in a graph are used to enable the GNNs to utilize the homophily relational data. However, not all graphs are homophilic, even in the same graph, the distributions may vary significantly. Using the same convolution over all nodes may lead to the ignorance of various graph patterns. Furthermore, many existing GNNs integrate node features and structure ide…
Institute of Cyberspace Security, Zhejiang University of Technology（浙江工业大学网络空间安全学院）；Binjiang Institute of Artificial Intelligence（滨江人工智能研究院）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；Science and Technology on Communication Information Security Control Laboratory（通信信息安全控制科学与技术实验室）
Renmin University of China（中国人民大学）；Alibaba Group（阿里巴巴集团）
Graph Neural Networks (GNNs) have received extensive research attention for their promising performance in graph machine learning. Despite their extraordinary predictive accuracy, existing approaches, such as GCN and GPRGNN, are not robust in the face of homophily changes on test graphs, rendering these models vulnerable to graph structural attacks and with limited capacity in generalizing to graphs of varied homophily levels. Although many methods have been proposed to improve the robustness of GNN models, most of these techniques are restricted to the spatial domain and employ complicated de…
LSGNN: Towards General Graph Neural Network in Node Classification by Local Similarity
Sun Yat-sen University（中山大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Hebei University of Technology（河北工业大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Adapting to Heterophilic Graph Data with Structure-Guided Neighbor Discovery
Department of Signal Theory and Communications, King Juan Carlos University（信号理论与通信系，国王胡安·卡洛斯大学）；Department of Electrical and Computer Engineering, Rice University（电气与计算机工程系，里奇大学）
Message passing is a fundamental procedure for graph neural networks in the field of graph representation learning. Based on the homophily assumption, the current message passing always aggregates features of connected nodes, such as the graph Laplacian smoothing process. However, real-world graphs tend to be noisy and/or non-smooth. The homophily assumption does not always hold, leading to sub-optimal results. A revised message passing method needs to maintain each node's discriminative ability when aggregating the message from neighbors. To this end, we propose a Memory-based Message Passing…
Graph Neural Networks (GNNs) extend basic Neural Networks (NNs) by using graph structures based on the relational inductive bias (homophily assumption). While GNNs have been commonly believed to outperform NNs in real-world tasks, recent work has identified a non-trivial set of datasets where their performance compared to NNs is not satisfactory. Heterophily has been considered the main cause of this empirical observation and numerous works have been put forward to address it. In this paper, we first revisit the widely used homophily metrics and point out that their consideration of only graph…
Attention mechanism in graph neural networks is designed to assign larger weights to important neighbor nodes for better representation. However, what graph attention learns is not understood well, particularly when graphs are noisy. In this paper, we propose a self-supervised graph attention network (SuperGAT), an improved graph attention model for noisy graphs. Specifically, we exploit two attention forms compatible with a self-supervised task to predict edges, whose presence and absence contain the inherent information about the importance of the relationships between nodes. By encoding edg…
Xi'an Jiaotong University（西安交通大学）；Xi'an University of Science and Technology（西安科技大学）
In representation learning on the graph-structured data, under heterophily (or low homophily), many popular GNNs may fail to capture long-range dependencies, which leads to their performance degradation. To solve the above-mentioned issue, we propose a graph convolutional networks with structure learning (GCN-SL), and furthermore, the proposed approach can be applied to node classification. The proposed GCN-SL contains two improvements: corresponding to node features and edges, respectively. In the aspect of node features, we propose an efficient-spectral-clustering (ESC) and an ESC with ancho…
Explicitly or implicitly, most of dimensionality reduction methods need to determine which samples are neighbors and the similarity between the neighbors in the original highdimensional space. The projection matrix is then learned on the assumption that the neighborhood information (e.g., the similarity) is known and fixed prior to learning. However, it is difficult to precisely measure the intrinsic similarity of samples in high-dimensional space because of the curse of dimensionality. Consequently, the neighbors selected according to such similarity might and the projection matrix obtained a…
Beckman Institute, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校贝克曼研究所）；Adobe Research（奥多比研究院）
Classifying large-scale image data into object categories is an important problem that has received increasing research attention. Given the huge amount of data, non-parametric approaches such as nearest neighbor classifiers have shown promising results, especially when they are underpinned by a learned distance or similarity measurement. Although metric learning has been well studied in the past decades, most existing algorithms are impractical to handle large-scale data sets. In this paper, we present an image similarity learning method that can scale well in both the number of images and th…
The Australian National University（澳大利亚国立大学）；Pohang University of Science and Technology（浦项科技大学）
Graph neural networks (GNNs) have been extensively studied for prediction tasks on graphs. As pointed out by recent studies, most GNNs assume local homophily, i.e., strong similarities in local neighborhoods. This assumption however limits the generalizability power of GNNs. To address this limitation, we propose a flexible GNN model, which is capable of handling any graphs without being restricted by their underlying homophily. At its core, this model adopts a node attention mechanism based on multiple learnable spectral filters; therefore, the aggregation scheme is learned adaptively for eac…
Graph Neural Networks (GNNs) achieve state-of-the-art performance on graph-structured data across numerous domains. Their underlying ability to represent nodes as summaries of their vicinities has proven effective for homophilous graphs in particular, in which same-type nodes tend to connect. On heterophilous graphs, in which different-type nodes are likely connected, GNNs perform less consistently, as neighborhood information might be less representative or even misleading. On the other hand, GNN performance is not inferior on all heterophilous graphs, and there is a lack of understanding of …
This technical note compares two coding (quantization) schemes for random projections in the context of sub-linear time approximate near neighbor search. The first scheme is based on uniform quantization while the second scheme utilizes a uniform quantization plus a uniformly random offset (which has been popular in practice). The prior work compared the two schemes in the context of similarity estimation and training linear classifiers, with the conclusion that the step of random offset is not necessary and may hurt the performance (depending on the similarity level). The task of near neighbo…
