Relation prediction for knowledge graphs aims at predicting missing relationships between entities. Despite the importance of inductive relation prediction, most previous works are limited to a transductive setting and cannot process previously unseen entities. The recent proposed subgraph-based relation reasoning models provided alternatives to predict links from the subgraph structure surrounding a candidate triplet inductively. However, we observe that these methods often neglect the directed nature of the extracted subgraph and weaken the role of relation information in the subgraph modeling. As a result, they fail to effectively handle the asymmetric/anti-symmetric triplets and produce insufficient embeddings for the target triplets. To this end, we introduce a \textbf{C}\textbf{o}mmunicative \textbf{M}essage \textbf{P}assing neural network for \textbf{I}nductive re\textbf{L}ation r\textbf{E}asoning, \textbf{CoMPILE}, that reasons over local directed subgraph structures and has a vigorous inductive bias to process entity-independent semantic relations. In contrast to existing models, CoMPILE strengthens the message interactions between edges and entitles through a communicative kernel and enables a sufficient flow of relation information. Moreover, we demonstrate that CoMPILE can naturally handle asymmetric/anti-symmetric relations without the need for explosively increasing the number of model parameters by extracting the directed enclosing subgraphs. Extensive experiments show substantial performance gains in comparison to state-of-the-art methods on commonly used benchmark datasets with variant inductive settings.
Zhejiang University（浙江大学）；Donghai Laboratory（东海实验室）；The University of Manchester（曼彻斯特大学）；The University of Edinburgh（爱丁堡大学）；Huawei Technologies Co., Ltd.（华为技术有限公司）
In knowledge graph completion (KGC), predicting triples involving emerging entities and/or relations, which are unseen when the KG embeddings are learned, has become a critical challenge. Subgraph reasoning with message passing is a promising and popular solution. Some recent methods have achieved good performance, but they (i) usually can only predict triples involving unseen entities alone, failing to address more realistic fully inductive situations with both unseen entities and unseen relations, and (ii) often conduct message passing over the entities with the relation patterns not fully u…
Sun Yat-sen University（中山大学）；Galixir Technologies Ltd（Galixir科技有限公司）
Constructing appropriate representations of molecules lies at the core of numerous tasks such as material science, chemistry and drug designs. Recent researches abstract molecules as attributed graphs and employ graph neural networks (GNN) for molecular representation learning, which have made remarkable achievements in molecular graph modeling. Albeit powerful, current models either are based on local aggregation operations and thus miss higher-order graph properties or focus on only node information without fully using the edge information. For this sake, we propose a Communicative Message P…
Expanding the Scope: Inductive Knowledge Graph Reasoning with Multi-Starting Progressive Propagation
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；National Institute of Healthcare Data Science, Nanjing University（南京大学健康医疗大数据国家研究院）
Message Intercommunication for Inductive Relation Reasoning
School of Computer, National University of Defense Technology（国防科技大学计算机学院）；College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学学院）
Graph Inductive Biases in Transformers without Message Passing
McGill University（麦吉尔大学）；University of Oxford（牛津大学）；Massachusetts Institute of Technology（麻省理工学院）；MetaAI（Meta人工智能实验室）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Canada CIFAR AI Chair（加拿大CIFAR人工智能讲席）；FiveAI（FiveAI公司）；International Laboratory on Learning Systems（国际学习系统实验室）
本文提出 GRIT，通过相对位置编码、节点对注意力和度信息注入，在无消息传递的情况下赋予 Graph Transformer 图归纳偏置，并取得理论表达力与 SOTA 性能。
Shanxi University（山西大学）；Cardiff University（卡迪夫大学）；A*STAR（新加坡科技研究局）；University of Edinburgh（爱丁堡大学）
Multi-hop reasoning over real-life knowledge graphs (KGs) is a highly challenging problem as traditional subgraph matching methods are not capable to deal with noise and missing information. To address this problem, it has been recently introduced a promising approach based on jointly embedding logical queries and KGs into a low-dimensional space to identify answer entities. However, existing proposals ignore critical semantic knowledge inherently available in KGs, such as type information. To leverage type information, we propose a novel TypE-aware Message Passing (TEMP) model, which enhances…
Fudan University（复旦大学）；The University of Sydney（悉尼大学）
The message-passing mechanism helps Graph Neural Networks (GNNs) achieve remarkable results on various node classification tasks. Nevertheless, the recursive nodes fetching and aggregation in message-passing cause inference latency when deploying GNNs to large-scale graphs. One promising inference acceleration direction is to distill the GNNs into message-passing-free student multi-layer perceptrons (MLPs). However, the MLP student cannot fully learn the structure knowledge due to the lack of structure inputs, which causes inferior performance in the heterophily and inductive scenarios. To add…
A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；National Institute of Healthcare Data Science, Nanjing University（南京大学国家卫生数据科学研究院）
Dual-channel Heterophilic Message Passing for Graph Fraud Detection
University of Chinese Academy of Science（中国科学院大学）；Fuzhou University（福州大学）；Northeast Normal University（东北师范大学）；Hainan University（海南大学）；Tsinghua University（清华大学）；The Australian National University（澳大利亚国立大学）
Learning Rule-Induced Subgraph Representations for Inductive Relation Prediction
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
This work provides the first theoretical study on the ability of graph Message Passing Neural Networks (gMPNNs) -- such as Graph Neural Networks (GNNs) -- to perform inductive out-of-distribution (OOD) link prediction tasks, where deployment (test) graph sizes are larger than training graphs. We first prove non-asymptotic bounds showing that link predictors based on permutation-equivariant (structural) node embeddings obtained by gMPNNs can converge to a random guess as test graphs get larger. We then propose a theoretically-sound gMPNN that outputs structural pairwise (2-node) embeddings and …
Graph neural networks (GNNs) are a powerful inductive bias for modelling algorithmic reasoning procedures and data structures. Their prowess was mainly demonstrated on tasks featuring Markovian dynamics, where querying any associated data structure depends only on its latest state. For many tasks of interest, however, it may be highly beneficial to support efficient data structure queries dependent on previous states. This requires tracking the data structure's evolution through time, placing significant pressure on the GNN's latent representations. We introduce Persistent Message Passing (PMP…
The University of Edinburgh（爱丁堡大学）；Google Research（谷歌研究院）；Google（谷歌）；Amazon Japan（亚马逊日本）
In this work we propose Pathfinder Discovery Networks (PDNs), a method for jointly learning a message passing graph over a multiplex network with a downstream semi-supervised model. PDNs inductively learn an aggregated weight for each edge, optimized to produce the best outcome for the downstream learning task. PDNs are a generalization of attention mechanisms on graphs which allow flexible construction of similarity functions between nodes, edge convolutions, and cheap multiscale mixing layers. We show that PDNs overcome weaknesses of existing methods for graph attention (e.g. Graph Attention…
Feinberg School of Medicine（范伯格医学院）；Northwestern University（西北大学）
Hypergraph neural networks can model multi-way connections among nodes of the graphs, which are common in real-world applications such as genetic medicine. In particular, genetic pathways or gene sets encode molecular functions driven by multiple genes, naturally represented as hyperedges. Thus, hypergraph-guided embedding can capture functional relations in learned representations. Existing hypergraph neural network models often focus on node-level or graph-level inference. There is an unmet need in learning powerful representations of subgraphs of hypergraphs in real-world applications. For …
Inference of Sequential Patterns for Neural Message Passing in Temporal Graphs
University of Zurich（苏黎世大学）；Julius-Maximilians-Universität Würzburg（维尔茨堡大学）；Center for Artificial Intelligence and Data Science (CAIDAS)（人工智能与数据科学中心）；Chair of Machine Learning for Complex Networks（复杂网络机器学习主席席位）
Message passing Graph Neural Networks (GNNs) provide a powerful modeling framework for relational data. However, the expressive power of existing GNNs is upper-bounded by the 1-Weisfeiler-Lehman (1-WL) graph isomorphism test, which means GNNs that are not able to predict node clustering coefficients and shortest path distances, and cannot differentiate between different d-regular graphs. Here we develop a class of message passing GNNs, named Identity-aware Graph Neural Networks (ID-GNNs), with greater expressive power than the 1-WL test. ID-GNN offers a minimal but powerful solution to limitat…
MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；IBM Research（IBM研究院）
Graph-structured data ubiquitously appears in science and engineering. Graph neural networks (GNNs) are designed to exploit the relational inductive bias exhibited in graphs; they have been shown to outperform other forms of neural networks in scenarios where structure information supplements node features. The most common GNN architecture aggregates information from neighborhoods based on message passing. Its generality has made it broadly applicable. In this paper, we focus on a special, yet widely used, type of graphs -- DAGs -- and inject a stronger inductive bias -- partial ordering -- in…
MMGNN: Multi-level, multi-color graph neural networks for molecular property prediction
The Bredesen Center, University of Tennessee, Knoxville（田纳西大学诺克斯维尔分校布雷德森中心）；Department of Mathematics, University of Tennessee, Knoxville（田纳西大学诺克斯维尔分校数学系）
Hulu Innovation Lab（Hulu创新实验室）；Peking University（北京大学）
Real-world scenarios demand reasoning about process, more than final outcome prediction, to discover latent causal chains and better understand complex systems. It requires the learning algorithms to offer both accurate predictions and clear interpretations. We design a set of trajectory reasoning tasks on graphs with only the source and the destination observed. We present the attention flow mechanism to explicitly model the reasoning process, leveraging the relational inductive biases by basing our models on graph networks. We study the way attention flow can effectively act on the underlyin…
