Higher-order proximity (HOP) is fundamental for most network embedding methods due to its significant effects on the quality of node embedding and performance on downstream network analysis tasks. Most existing HOP definitions are based on either homophily to place close and highly interconnected nodes tightly in embedding space or heterophily to place distant but structurally similar nodes together after embedding. In real-world networks, both can co-exist, and thus considering only one could limit the prediction performance and interpretability. However, there is no general and universal solution that takes both into consideration. In this paper, we propose such a simple yet powerful framework called homophily and heterophliy preserving network transformation (H2NT) to capture HOP that flexibly unifies homophily and heterophily. Specifically, H2NT utilises motif representations to transform a network into a new network with a hybrid assumption via micro-level and macro-level walk paths. H2NT can be used as an enhancer to be integrated with any existing network embedding methods without requiring any changes to latter methods. Because H2NT can sparsify networks with motif structures, it can also improve the computational efficiency of existing network embedding methods when integrated. We conduct experiments on node classification, structural role classification and motif prediction to show the superior prediction performance and computational efficiency over state-of-the-art methods. In particular, DeepWalk-based H2 NT achieves 24% improvement in terms of precision on motif prediction, while reducing 46% computational time compared to the original DeepWalk.
Graph Neural Networks (GNNs) have achieved remarkable success in diverse real-world applications. Traditional GNNs are designed based on homophily, which leads to poor performance under heterophily scenarios. Current solutions deal with heterophily mainly by mixing high-order neighbors or passing signed messages. However, mixing high-order neighbors destroys the original graph structure and passing signed messages utilizes an inflexible message-passing mechanism, which is prone to producing unsatisfactory effects. To overcome the above problems, we propose a novel GNN model based on relation v…
University of Michigan（密歇根大学）；Adobe Research（Adobe研究院）；Intel Labs（英特尔实验室）
Graph Neural Networks (GNNs) have proven to be useful for many different practical applications. However, many existing GNN models have implicitly assumed homophily among the nodes connected in the graph, and therefore have largely overlooked the important setting of heterophily, where most connected nodes are from different classes. In this work, we propose a novel framework called CPGNN that generalizes GNNs for graphs with either homophily or heterophily. The proposed framework incorporates an interpretable compatibility matrix for modeling the heterophily or homophily level in the graph, w…
Graph Neural Networks (GNNs) extend basic Neural Networks (NNs) by using graph structures based on the relational inductive bias (homophily assumption). While GNNs have been commonly believed to outperform NNs in real-world tasks, recent work has identified a non-trivial set of datasets where their performance compared to NNs is not satisfactory. Heterophily has been considered the main cause of this empirical observation and numerous works have been put forward to address it. In this paper, we first revisit the widely used homophily metrics and point out that their consideration of only graph…
We investigate the long-time properties of a dynamic, out-of-equilibrium network of individuals holding one of two opinions in a population consisting of two communities of different sizes. Here, while the agents' opinions are fixed, they have a preferred degree which leads them to endlessly create and delete links. Our evolving network is shaped by homophily/heterophily, which is a form of social interaction by which individuals tend to establish links with others having similar/dissimilar opinions. Using Monte Carlo simulations and a detailed mean-field analysis, we study in detail how the s…
Graph Neural Networks (GNNs) extend basic Neural Networks (NNs) by using the graph structures based on the relational inductive bias (homophily assumption). Though GNNs are believed to outperform NNs in real-world tasks, performance advantages of GNNs over graph-agnostic NNs seem not generally satisfactory. Heterophily has been considered as a main cause and numerous works have been put forward to address it. In this paper, we first show that not all cases of heterophily are harmful for GNNs with aggregation operation. Then, we propose new metrics based on a similarity matrix which considers t…
Beihang University（北京航空航天大学）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Shen Yuan Honors College, Beihang University（北京航空航天大学沈元荣誉学院）；Hebei University of Technology（河北工业大学）；School of Artificial Intelligence, Hebei University of Technology（河北工业大学人工智能学院）
Unsupervised graph representation learning (UGRL) has drawn increasing research attention and achieved promising results in several graph analytic tasks. Relying on the homophily assumption, existing UGRL methods tend to smooth the learned node representations along all edges, ignoring the existence of heterophilic edges that connect nodes with distinct attributes. As a result, current methods are hard to generalize to heterophilic graphs where dissimilar nodes are widely connected, and also vulnerable to adversarial attacks. To address this issue, we propose a novel unsupervised Graph Represe…
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；School of Cyberspace, Hangzhou Dianzi University（杭州电子科技大学网络空间安全学院）
Graph Convolutional Network (GCN) has shown remarkable potential of exploring graph representation. However, the GCN aggregating mechanism fails to generalize to networks with heterophily where most nodes have neighbors from different classes, which commonly exists in real-world networks. In order to make the propagation and aggregation mechanism of GCN suitable for both homophily and heterophily (or even their mixture), we introduce block modeling into the framework of GCN so that it can realize "block-guided classified aggregation", and automatically learn the corresponding aggregation rules…
Institute of Cyberspace Security, Zhejiang University of Technology（浙江工业大学网络空间安全学院）；Binjiang Institute of Artificial Intelligence（滨江人工智能研究院）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；Science and Technology on Communication Information Security Control Laboratory（通信信息安全控制科学与技术实验室）
University of Luxembourg（卢森堡大学）；Imperial College London（伦敦帝国理工学院）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Network embedding (NE) approaches have emerged as a predominant technique to represent complex networks and have benefited numerous tasks. However, most NE approaches rely on a homophily assumption to learn embeddings with the guidance of supervisory signals, leaving the unsupervised heterophilous scenario relatively unexplored. This problem becomes especially relevant in fields where a scarcity of labels exists. Here, we formulate the unsupervised NE task as an r-ego network discrimination problem and develop the SELENE framework for learning on networks with homophily and heterophily. Specif…
How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashing
School of Computing, National University of Singapore（新加坡国立大学计算机学院）；Zhangjiang Institute for Advanced Study, Shanghai Jiao Tong University（上海交通大学张江高等研究院）；Shanghai AI Laboratory（上海人工智能实验室）；School of Mathematics and Statistics, University of New South Wales（新南威尔士大学数学与统计学院）；Zhejiang Institute of Optoelectronics（浙江光电研究院）；Zhejiang Normal University（浙江师范大学）；Department of Computer Science and Technology, Cambridge University（剑桥大学计算机科学与技术系）
Recent work has shown that a simple, fast method called Simple Graph Convolution (SGC) (Wu et al., 2019), which eschews deep learning, is competitive with deep methods like graph convolutional networks (GCNs) (Kipf & Welling, 2017) in common graph machine learning benchmarks. The use of graph data in SGC implicitly assumes the common but not universal graph characteristic of homophily, wherein nodes link to nodes which are similar. Here we confirm that SGC is indeed ineffective for heterophilous (i.e., non-homophilous) graphs via experiments on synthetic and real-world datasets. We propose Ada…
Graph Neural Networks (GNNs) have been predominant for graph learning tasks; however, recent studies showed that a well-known graph algorithm, Label Propagation (LP), combined with a shallow neural network can achieve comparable performance to GNNs in semi-supervised node classification on graphs with high homophily. In this paper, we show that this approach falls short on graphs with low homophily, where nodes often connect to the nodes of the opposite classes. To overcome this, we carefully design a combination of a base predictor with LP algorithm that enjoys a closed-form solution as well …
Univ. of Southern California（南加州大学）；Fudan University（复旦大学）；Information Sciences Institute（信息科学研究所）
In this paper, for the first time, we study label propagation in heterogeneous graphs under heterophily assumption. Homophily label propagation (i.e., two connected nodes share similar labels) in homogeneous graph (with same types of vertices and relations) has been extensively studied before. Unfortunately, real-life networks are heterogeneous, they contain different types of vertices (e.g., users, images, texts) and relations (e.g., friendships, co-tagging) and allow for each node to propagate both the same and opposite copy of labels to its neighbors. We propose a $\mathcal{K}$-partite labe…
Monash University（莫纳什大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Shenzhen University（深圳大学）；Chinese Academy of Sciences（中国科学院）；Griffith University（格里菲斯大学）
Graph neural architecture search (NAS) has gained popularity in automatically designing powerful graph neural networks (GNNs) with relieving human efforts. However, existing graph NAS methods mainly work under the homophily assumption and overlook another important graph property, i.e., heterophily, which exists widely in various real-world applications. To date, automated heterophilic graph learning with NAS is still a research blank to be filled in. Due to the complexity and variety of heterophilic graphs, the critical challenge of heterophilic graph NAS mainly lies in developing the heterop…
We consider a dynamic network of individuals that may hold one of two different opinions in a two-party society. As a dynamical model, agents can endlessly create and delete links to satisfy a preferred degree, and the network is shaped by \emph{homophily}, a form of social interaction. Characterized by the parameter $J \in [-1,1]$, the latter plays a role similar to Ising spins: agents create links to others of the same opinion with probability $(1+J)/2$, and delete them with probability $(1-J)/2$. Using Monte Carlo simulations and mean-field theory, we focus on the network structure in the s…
Electronic communication records provide detailed information about temporal aspects of human interaction. Previous studies have shown that individuals' communication patterns have complex temporal structure, and that this structure has system-wide effects. In this paper we use mobile phone records to show that interaction patterns involving multiple individuals have non-trivial temporal structure that cannot be deduced from a network presentation where only interaction frequencies are taken into account. We apply a recently introduced method, temporal motifs, to identify interaction patterns …
Constituents of complex systems interact with each other and self-organize to form complex networks. Empirical results show that the link formation process of many real networks follows either the global principle of popularity or the local principle of similarity or a tradeoff between the two. In particular, it has been shown that in social networks individuals exhibit significant homophily when choosing their collaborators. We demonstrate, however, that in populations in which there is a division of labor, skill complementarity is an important factor in the formation of socioeconomic network…
The statistical significance of network properties is conditioned on null models which satisfy spec- ified properties but that are otherwise random. Exponential random graph models are a principled theoretical framework to generate such constrained ensembles, but which often fail in practice, either due to model inconsistency, or due to the impossibility to sample networks from them. These problems affect the important case of networks with prescribed clustering coefficient or number of small connected subgraphs (motifs). In this paper we use the Wang-Landau method to obtain a multicanonical s…
Self-Attention Empowered Graph Convolutional Network for Structure Learning and Node Embedding
School of Electronic and Information Engineering, Xi’an Jiaotong University（西安交通大学电子与信息工程学院）；College of Geomatics, Xi’an University of Science and Technology（西安科技大学测绘学院）
It is known that individuals in social networks tend to exhibit homophily (a.k.a. assortative mixing) in their social ties, which implies that they prefer bonding with others of their own kind. But what are the reasons for this phenomenon? Is it that such relations are more convenient and easier to maintain? Or are there also some more tangible benefits to be gained from this collective behaviour? The current work takes a game-theoretic perspective on this phenomenon, and studies the conditions under which different assortative mixing strategies lead to equilibrium in an evolving social networ…
The principle of similarity, or homophily, is often used to explain patterns observed in complex networks such as transitivity and the abundance of triangles (3-cycles). However, many phenomena from division of labor to protein-protein interactions (PPI) are driven by complementarity (differences and synergy). Here we show that the principle of complementarity is linked to the abundance of quadrangles (4-cycles) and dense bipartite-like subgraphs. We link both principles to their characteristic motifs and introduce two families of coefficients of: (1) structural similarity, which generalize lo…
