Community is a common characteristic of networks including social networks, biological networks, computer and information networks, to name a few. Community detection is a basic step for exploring and analysing these network data. Typically, homogenous network is a type of networks which consists of only one type of objects with one type of links connecting them. There has been a large body of developments in models and algorithms to detect communities over it. However, real-world networks naturally exhibit heterogeneous qualities appearing as multiple types of objects with multi-relational links connecting them. Those heterogeneous information could facilitate the community detection for its constituent homogeneous networks, but has not been fully explored. In this paper, we exploit heterogeneous multi-relational networks (HMRNet) and propose an efficient message passing based algorithm to simultaneously detect communities for all homogeneous networks. Specifically, an HMRNet is reorganized into a hierarchical structure with homogeneous networks as its layers and heterogeneous links connecting them. To detect communities in such an HMRNet, the problem is formulated as a maximum a posterior (MAP) over a factor graph. Finally a message passing based algorithm is derived to find a best solution of the MAP problem. Evaluation on both synthetic and real-world networks confirms the effectiveness of the proposed method.
Knowledge graph completion aims to predict missing relations between entities in a knowledge graph. In this work, we propose a relational message passing method for knowledge graph completion. Different from existing embedding-based methods, relational message passing only considers edge features (i.e., relation types) without entity IDs in the knowledge graph, and passes relational messages among edges iteratively to aggregate neighborhood information. Specifically, two kinds of neighborhood topology are modeled for a given entity pair under the relational message passing framework: (1) Relat…
Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing
Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克数学研究所）；UCLA（加州大学洛杉矶分校）；Center for Scalable Data Analytics and Artificial Intelligence（可扩展数据与人工智能中心）；University of Oxford（牛津大学）
TU Dresden（德累斯顿工业大学）；Fraunhofer IAIS（弗劳恩霍夫应用信息学研究所）；University of Bonn（波恩大学）
Hyper-relational knowledge graphs (KGs) (e.g., Wikidata) enable associating additional key-value pairs along with the main triple to disambiguate, or restrict the validity of a fact. In this work, we propose a message passing based graph encoder - StarE capable of modeling such hyper-relational KGs. Unlike existing approaches, StarE can encode an arbitrary number of additional information (qualifiers) along with the main triple while keeping the semantic roles of qualifiers and triples intact. We also demonstrate that existing benchmarks for evaluating link prediction (LP) performance on hyper…
Modern methods for learning over graph input data have shown the fruitfulness of accounting for relationships among elements in a collection. However, most methods that learn over set input data use only rudimentary approaches to exploit intra-collection relationships. In this work we introduce Deep Message Passing on Sets (DMPS), a novel method that incorporates relational learning for sets. DMPS not only connects learning on graphs with learning on sets via deep kernel learning, but it also bridges message passing on sets and traditional diffusion dynamics commonly used in denoising models. …
Neural Processes (NPs) are powerful and flexible models able to incorporate uncertainty when representing stochastic processes, while maintaining a linear time complexity. However, NPs produce a latent description by aggregating independent representations of context points and lack the ability to exploit relational information present in many datasets. This renders NPs ineffective in settings where the stochastic process is primarily governed by neighbourhood rules, such as cellular automata (CA), and limits performance for any task where relational information remains unused. We address this…
We introduce the nonparametric metadata dependent relational (NMDR) model, a Bayesian nonparametric stochastic block model for network data. The NMDR allows the entities associated with each node to have mixed membership in an unbounded collection of latent communities. Learned regression models allow these memberships to depend on, and be predicted from, arbitrary node metadata. We develop efficient MCMC algorithms for learning NMDR models from partially observed node relationships. Retrospective MCMC methods allow our sampler to work directly with the infinite stick-breaking representation o…
Many complex processes can be viewed as dynamical systems of interacting agents. In many cases, only the state sequences of individual agents are observed, while the interacting relations and the dynamical rules are unknown. The neural relational inference (NRI) model adopts graph neural networks that pass messages over a latent graph to jointly learn the relations and the dynamics based on the observed data. However, NRI infers the relations independently and suffers from error accumulation in multi-step prediction at dynamics learning procedure. Besides, relation reconstruction without prior…
Tsinghua University（清华大学）；Tencent（腾讯）；Carnegie Mellon University（卡内基梅隆大学）；Frontier Science Center for Quantum Information（量子信息前沿科学中心）；RIKEN Center for Emergent Matter Science（理化学研究所新兴物质科学中心）
With many frameworks based on message passing neural networks proposed to predict molecular and bulk properties, machine learning methods have tremendously shifted the paradigms of computational sciences underpinning physics, material science, chemistry, and biology. While existing machine learning models have yielded superior performances in many occasions, most of them model and process molecular systems in terms of homogeneous graph, which severely limits the expressive power for representing diverse interactions. In practice, graph data with multiple node and edge types is ubiquitous and m…
Graph Neural Networks (GNNs) are the subject of intense focus by the machine learning community for problems involving relational reasoning. GNNs can be broadly divided into spatial and spectral approaches. Spatial approaches use a form of learned message-passing, in which interactions among vertices are computed locally, and information propagates over longer distances on the graph with greater numbers of message-passing steps. Spectral approaches use eigendecompositions of the graph Laplacian to produce a generalization of spatial convolutions to graph structured data which access informatio…
Graph neural networks (GNNs) are emerging machine learning models on graphs. Permutation-equivariance and proximity-awareness are two important properties highly desirable for GNNs. Both properties are needed to tackle some challenging graph problems, such as finding communities and leaders. In this paper, we first analytically show that the existing GNNs, mostly based on the message-passing mechanism, cannot simultaneously preserve the two properties. Then, we propose Stochastic Message Passing (SMP) model, a general and simple GNN to maintain both proximity-awareness and permutation-equivari…
The existing literature on knowledge graph completion mostly focuses on the link prediction task. However, knowledge graphs have an additional incompleteness problem: their nodes possess numerical attributes, whose values are often missing. Our approach, denoted as MrAP, imputes the values of missing attributes by propagating information across the multi-relational structure of a knowledge graph. It employs regression functions for predicting one node attribute from another depending on the relationship between the nodes and the type of the attributes. The propagation mechanism operates iterat…
University of Chinese Academy of Sciences（中国科学院大学）；University of Rochester（罗切斯特大学）；University of Science and Technology of China（中国科学技术大学）；Peng Cheng Laboratory（鹏城实验室）
Graph neural architecture search has sparked much attention as Graph Neural Networks (GNNs) have shown powerful reasoning capability in many relational tasks. However, the currently used graph search space overemphasizes learning node features and neglects mining hierarchical relational information. Moreover, due to diverse mechanisms in the message passing, the graph search space is much larger than that of CNNs. This hinders the straightforward application of classical search strategies for exploring complicated graph search space. We propose Automatic Relation-aware Graph Network Proliferat…
Beijing University of Posts and Telecommunications（北京邮电大学）；Meituan（美团）
It is well established that graph neural networks (GNNs) can be interpreted and designed from the perspective of optimization objective. With this clear optimization objective, the deduced GNNs architecture has sound theoretical foundation, which is able to flexibly remedy the weakness of GNNs. However, this optimization objective is only proved for GNNs with single-relational graph. Can we infer a new type of GNNs for multi-relational graphs by extending this optimization objective, so as to simultaneously solve the issues in previous multi-relational GNNs, e.g., over-parameterization? In thi…
Graph Neural Networks (GNNs) have emerged as a powerful technique for learning on relational data. Owing to the relatively limited number of message passing steps they perform -- and hence a smaller receptive field -- there has been significant interest in improving their expressivity by incorporating structural aspects of the underlying graph. In this paper, we explore the use of affinity measures as features in graph neural networks, in particular measures arising from random walks, including effective resistance, hitting and commute times. We propose message passing networks based on these …
Multi-relational learning has received lots of attention from researchers in various research communities. Most existing methods either suffer from superlinear per-iteration cost, or are sensitive to the given ranks. To address both issues, we propose a scalable core tensor trace norm Regularized Orthogonal Iteration Decomposition (ROID) method for full or incomplete tensor analytics, which can be generalized as a graph Laplacian regularized version by using auxiliary information or a sparse higher-order orthogonal iteration (SHOOI) version. We first induce the equivalence relation of the Scha…
Most work in the area of statistical relational learning (SRL) is focussed on discrete data, even though a few approaches for hybrid SRL models have been proposed that combine numerical and discrete variables. In this paper we distinguish numerical random variables for which a probability distribution is defined by the model from numerical input variables that are only used for conditioning the distribution of discrete response variables. We show how numerical input relations can very easily be used in the Relational Bayesian Network framework, and that existing inference and learning methods …
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This paper addresses the question of identifying the best candidate answer to a question on Community Question Answer (CQA) forums. The problem is important because Individuals often visit CQA forums to seek answers to nuanced questions. We develop a novel induced relational graph convolutional network (IR-GCN) framework to address the question. We make three contributions. First, we introduce a modular framework that separates the construction of the graph with the label selection mechanism. We use equivalence relations to induce a graph comprising cliques and identify two label assignment me…
Graph neural networks (GNNs) are able to leverage the structure of graph data by passing messages along the edges of the graph. While this allows GNNs to learn features depending on the graph structure, for certain graph topologies it leads to inefficient information propagation and a problem known as oversquashing. This has recently been linked with the curvature and spectral gap of the graph. On the other hand, adding edges to the message-passing graph can lead to increasingly similar node representations and a problem known as oversmoothing. We propose a computationally efficient algorithm …
Uber Advanced Technologies Group（优步先进技术集团）；University of Toronto（多伦多大学）
In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor states via a message passing process. Inspired by Gaussian belief propagation, we design the messages to be spatially-transformed parameters of the output distributions from neighboring agents. Our m…
Federal University of Rio Grande do Sul（南里奥格兰德联邦大学）；University of Siena（锡耶纳大学）
Recently, the deep learning community has given growing attention to neural architectures engineered to learn problems in relational domains. Convolutional Neural Networks employ parameter sharing over the image domain, tying the weights of neural connections on a grid topology and thus enforcing the learning of a number of convolutional kernels. By instantiating trainable neural modules and assembling them in varied configurations (apart from grids), one can enforce parameter sharing over graphs, yielding models which can effectively be fed with relational data. In this context, vertices in a…
Dynamic demand prediction is crucial for the efficient operation and management of urban transportation systems. Extensive research has been conducted on single-mode demand prediction, ignoring the fact that the demands for different transportation modes can be correlated with each other. Despite some recent efforts, existing approaches to multimodal demand prediction are generally not flexible enough to account for multiplex networks with diverse spatial units and heterogeneous spatiotemporal correlations across different modes. To tackle these issues, this study proposes a multi-relational s…
Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；Meta AI (FAIR)（Meta人工智能（基础人工智能研究团队））；Shanghai Jiao Tong University（上海交通大学）；Université de Montréal（蒙特利尔大学）；HEC Montréal（蒙特利尔高等商学院）
Modeling spatial relationship in the data remains critical across many different tasks, such as image classification, semantic segmentation and protein structure understanding. Previous works often use a unified solution like relative positional encoding. However, there exists different kinds of spatial relations, including short-range, medium-range and long-range relations, and modeling them separately can better capture the focus of different tasks on the multi-range relations (e.g., short-range relations can be important in instance segmentation, while long-range relations should be upweigh…
Sun Yat-sen University（中山大学）；Galixir Technologies Ltd（Galixir科技有限公司）
Illuminating the interconnections between drugs and genes is an important topic in drug development and precision medicine. Currently, computational predictions of drug-gene interactions mainly focus on the binding interactions without considering other relation types like agonist, antagonist, etc. In addition, existing methods either heavily rely on high-quality domain features or are intrinsically transductive, which limits the capacity of models to generalize to drugs/genes that lack external information or are unseen during the training process. To address these problems, we propose a nove…
Ghent University - imec（根特大学-imec）；IDLab（互联网与数据实验室）
The inception of the Relational Graph Convolutional Network (R-GCN) marked a milestone in the Semantic Web domain as a widely cited method that generalises end-to-end hierarchical representation learning to Knowledge Graphs (KGs). R-GCNs generate representations for nodes of interest by repeatedly aggregating parameterised, relation-specific transformations of their neighbours. However, in this paper, we argue that the the R-GCN's main contribution lies in this "message passing" paradigm, rather than the learned weights. To this end, we introduce the "Random Relational Graph Convolutional Netw…
Polarized Direct Cross-Attention Message Passing in GNNs for Machinery Fault Diagnosis
Department of Automation, College of Artificial Intelligence, China University of Petroleum (Beijing)（自动化系，人工智能学院，中国石油大学（北京））；Department of Safety Engineering, College of Safety and Ocean Engineering, China University of Petroleum (Beijing)（安全工程系，安全与海洋工程学院，中国石油大学（北京））
Effects of relational graph modularity and depth on the learning performance of neural networks
Department of Physics, Gyeongsang National University（庆尚国立大学物理系）；The Research Institute of Natural Science, Gyeongsang National University（庆尚国立大学自然科学研究所）；Future Convergence Technology Research Institute, Gyeongsang National University（庆尚国立大学未来融合技术研究所）
Hyperbolic Hypergraph Neural Networks for Multi-Relational Knowledge Hypergraph Representation
National Engineering Research Center for Big Data Technology and System（国家大数据技术与系统工程研究中心）；School of Computer Science and Technology（计算机科学与技术学院）；Huazhong University of Science and Technology（华中科技大学）
University of Amsterdam（阿姆斯特丹大学）；Indian Institute of Technology Dhanbad（印度理工学院丹巴德分校）；AIQ
Graphs are the most ubiquitous data structures for representing relational datasets and performing inferences in them. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations. This drawback is mitigated by hypergraphs, in which an edge can connect an arbitrary number of nodes. Most hypergraph learning approaches convert the hypergraph structure to that of a graph and then deploy existing geometric deep learning methods. This transformation leads to information loss, and sub-optimal exploitation of the hypergraph's expressive power…
ContextGNN goes to Elliot: Towards Benchmarking Relational Deep Learning for Static Link Prediction (aka Personalized Item Recommendation)
University of Barcelona（巴塞罗那大学）；University of Southern California（南加州大学）；Université Paris-Saclay（巴黎萨克雷大学）；CentraleSupélec（中央理工-高等电力学院）；Inria（法国国家信息与自动化研究所）
University of Cambridge（剑桥大学）；Twitter（推特）；Imperial College London（帝国理工学院）；University of New South Wales（新南威尔士大学）；Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克数学科学研究所）；Shanghai Jiao Tong University（上海交通大学）；University of California, Los Angeles（加州大学洛杉矶分校）
The pairwise interaction paradigm of graph machine learning has predominantly governed the modelling of relational systems. However, graphs alone cannot capture the multi-level interactions present in many complex systems and the expressive power of such schemes was proven to be limited. To overcome these limitations, we propose Message Passing Simplicial Networks (MPSNs), a class of models that perform message passing on simplicial complexes (SCs). To theoretically analyse the expressivity of our model we introduce a Simplicial Weisfeiler-Lehman (SWL) colouring procedure for distinguishing no…
Hierarchical Multi-Relational Graph Representation Learning for Large-Scale Prediction of Drug-Drug Interactions
School of Electronics and Information Engineering, Xi’an Jiaotong University（西安交通大学电子与信息工程学院）；College of Geomatics, Xi’an University of Science and Technology（西安科技大学测绘学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）
Existing message passing neural networks for heterogeneous graphs rely on the concepts of meta-paths or meta-graphs due to the intrinsic nature of heterogeneous graphs. However, the meta-paths and meta-graphs need to be pre-configured before learning and are highly dependent on expert knowledge to construct them. To tackle this challenge, we propose a novel concept of meta-node for message passing that can learn enriched relational knowledge from complex heterogeneous graphs without any meta-paths and meta-graphs by explicitly modeling the relations among the same type of nodes. Unlike meta-pa…
The application of graph neural networks (GNNs) to the domain of electrical power grids has high potential impact on smart grid monitoring. Even though there is a natural correspondence of power flow to message-passing in GNNs, their performance on power grids is not well-understood. We argue that there is a gap between GNN research driven by benchmarks which contain graphs that differ from power grids in several important aspects. Additionally, inductive learning of GNNs across multiple power grid topologies has not been explored with real-world data. We address this gap by means of (i) defin…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Institute for Humanities-Centered Artificial Intelligence, University of Hamburg（汉堡大学人文中心人工智能研究所）
Enhancing Graph Representations with Neighborhood-Contextualized Message-Passing
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）；Kyoto University（京都大学）；Ateneo de Manila University（马尼拉雅典耀大学）；UNI-President Information Philippines Corporation（统一信息菲律宾公司）；The Chinese University of Hong Kong（香港中文大学）
Message passing Graph Neural Networks (GNNs) provide a powerful modeling framework for relational data. However, the expressive power of existing GNNs is upper-bounded by the 1-Weisfeiler-Lehman (1-WL) graph isomorphism test, which means GNNs that are not able to predict node clustering coefficients and shortest path distances, and cannot differentiate between different d-regular graphs. Here we develop a class of message passing GNNs, named Identity-aware Graph Neural Networks (ID-GNNs), with greater expressive power than the 1-WL test. ID-GNN offers a minimal but powerful solution to limitat…
Knowledge-Integrated Representation Learning for Crypto Anomaly Detection under Extreme Label Scarcity; Relational Domain-Logic Integration with Retrieval-Grounded Context and Path-Level Explanations
AI and Business Analytics, Ewha Womans University, Seoul, Republic of Korea（爱媛女子大学人工智能与商务分析系）；Department of Business Administration, Kumoh National Institute of Technology, Gumi, Republic of Korea（Kumoh国立技术大学商业管理系）；Coretrustlink, Seoul, Republic of Korea（Coretrustlink）
Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing
Aston University（阿斯顿大学）；University of Cambridge（剑桥大学）；Koç University（科克大学）；Edith Cowan University（伊迪丝·考恩大学）；University of New South Wales（新南威尔士大学）；TU Berlin（柏林工业大学）
Scalable Message-Passing Quantum Graph Neural Networks in the Weisfeiler-Leman Hierarchy
LIP6, CNRS, Sorbonne Université, Paris, France（巴黎大学LIP6研究所、法国国家科学研究中心、索邦大学）；QC Ware, Palo Alto, USA and Paris, France（QC Ware公司，美国帕洛阿尔托及法国巴黎）；School of Informatics, University of Edinburgh, Edinburgh, UK（爱丁堡大学信息学院，英国爱丁堡）；Fujitsu Research of Europe Ltd., United Kingdom（日本富士通欧洲研究有限公司，英国）；Terra Quantum GmbH, Germany（德国Terra Quantum GmbH公司）；Quantum Research Center, Technology Innovation Institute, Abu Dhabi, UAE（阿布扎赫尔技术创新研究所量子研究中心，阿联酋）
Wireless Power Control Based on Large Language Models
School of Information Science and Engineering, Southeast University（东南大学信息科学与工程学院）；Department of Electrical and Computer Engineering, University of California, Santa Cruz（加州大学圣克鲁兹分校电气与计算机工程系）
