GNNs have been proven to perform highly effective in various node-level, edge-level, and graph-level prediction tasks in several domains. Existing approaches mainly focus on static graphs. However, many graphs change over time with their edge may disappear, or node/edge attribute may alter from one time to the other. It is essential to consider such evolution in representation learning of nodes in time varying graphs. In this paper, we propose a Temporal Multi-layered Position-aware Graph Neural Network (TMP-GNN), a node embedding approach for dynamic graph that incorporates the interdependence of temporal relations into embedding computation. We evaluate the performance of TMP-GNN on two different representations of temporal multilayered graphs. The performance is assessed against the most popular GNNs on node-level prediction tasks. Then, we incorporate TMP-GNN into a deep learning framework to estimate missing data and compare the performance with their corresponding competent GNNs from our former experiment, and a baseline method. Experimental results on four real-world datasets yield up to 58% of lower ROC AUC for pairwise node classification task, and 96% of lower MAE in missing feature estimation, particularly for graphs with a relatively high number of nodes and lower mean degree of connectivity.
University of Catania（卡塔尼亚大学）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
Networks are a powerful tool to model complex systems, and the definition of many Graph Neural Networks (GNN), Deep Learning algorithms that can handle networks, has opened a new way to approach many real-world problems that would be hardly or even untractable. In this paper, we propose mGNN, a framework meant to generalize GNNs to the case of multi-layer networks, i.e., networks that can model multiple kinds of interactions and relations between nodes. Our approach is general (i.e., not task specific) and has the advantage of extending any type of GNN without any computational overhead. We te…
Multilayer GNN for Predictive Maintenance and Clustering in Power Grids
organization= Industrial \& Manufacturing Engineering, North Dakota State University , addressline= NDSU Dept. 2485, PO Box 6050 , city= Fargo , postcode= 58108-6050 , state= ND , country= USA；organization= Department of Engineering Technology \& Construction Management, University of North Carolina at Charlotte , postcode= 28223 , state= NC , country= USA；organization= Department of Industrial \& Systems Engineering, North Carolina Agriculture \& Tech State University , city= Greensboro , postcode= 27411 , state= NC , country= USA；organization= Department of Industrial \& Management Systems Engineering, University of South Florida , addressline= 4202 East Fowler Avenue, ENG 030 , city= Tampa , postcode= 33620 , state= FL , country= USA
We explore the feasibility of combining Graph Neural Network-based policy architectures with Deep Reinforcement Learning as an approach to problems in systems. This fits particularly well with operations on networks, which naturally take the form of graphs. As a case study, we take the idea of data-driven routing in intradomain traffic engineering, whereby the routing of data in a network can be managed taking into account the data itself. The particular subproblem which we examine is minimising link congestion in networks using knowledge of historic traffic flows. We show through experiments …
Modern graph neural networks (GNNs) learn node embeddings through multilayer local aggregation and achieve great success in applications on assortative graphs. However, tasks on disassortative graphs usually require non-local aggregation. In addition, we find that local aggregation is even harmful for some disassortative graphs. In this work, we propose a simple yet effective non-local aggregation framework with an efficient attention-guided sorting for GNNs. Based on it, we develop various non-local GNNs. We perform thorough experiments to analyze disassortative graph datasets and evaluate ou…
Quantifying the Optimization and Generalization Advantages of Graph Neural Networks Over Multilayer Perceptrons
RIKEN AIP（日本学术振兴会先进研究所）；The University of Hong Kong（香港大学）；National University of Singapore（新加坡国立大学）；The University of New South Wales（新南威尔士大学）；University of Tokyo（东京大学）
University of Wolverhampton（伍尔弗汉普顿大学）；São Paulo State University（圣保罗州立大学）；TECNALIA（泰克纳利亚研究中心）；Basque Research & Technology Alliance（巴斯克研究与技术联盟）；University of the Basque Country（巴斯克大学）；Edinburgh Napier University（爱丁堡龙比亚大学）
This paper proposes a novel multimodal self-supervised architecture for energy-efficient audio-visual (AV) speech enhancement that integrates Graph Neural Networks with canonical correlation analysis (CCA-GNN). The proposed approach lays its foundations on a state-of-the-art CCA-GNN that learns representative embeddings by maximizing the correlation between pairs of augmented views of the same input while decorrelating disconnected features. The key idea of the conventional CCA-GNN involves discarding augmentation-variant information and preserving augmentation-invariant information while prev…
Dynamic Graph Neural Networks for Physiological Based Pharmacokinetic Modeling: A Novel Data Driven Approach to Drug Concentration Prediction
Independent Researcher（独立研究者）；University of Michigan -- Ann Arbor（密歇根大学安娜堡分校）；Texas A\&M University（德克萨斯农工大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of California -- Irvine（加州大学伊尔弗分校）
University of Oslo（奥斯陆大学）；Institute for Energy Systems（能源系统研究所）
With the increased penetration of wind energy into the power grid, it has become increasingly important to be able to predict the expected power production for larger wind farms. Deep learning (DL) models can learn complex patterns in the data and have found wide success in predicting wake losses and expected power production. This paper proposes a modular framework for attention-based graph neural networks (GNN), where attention can be applied to any desired component of a graph block. The results show that the model significantly outperforms a multilayer perceptron (MLP) and a bidirectional …
Massachusetts Institute of Technology (MIT)（麻省理工学院（MIT））；University of Maryland（马里兰大学）；University of Washington（华盛顿大学）；National Institute of Informatics (NII)（信息学研究所（NII））
We study how neural networks trained by gradient descent extrapolate, i.e., what they learn outside the support of the training distribution. Previous works report mixed empirical results when extrapolating with neural networks: while feedforward neural networks, a.k.a. multilayer perceptrons (MLPs), do not extrapolate well in certain simple tasks, Graph Neural Networks (GNNs) -- structured networks with MLP modules -- have shown some success in more complex tasks. Working towards a theoretical explanation, we identify conditions under which MLPs and GNNs extrapolate well. First, we quantify t…
University of Michigan（密歇根大学）；Carnegie Mellon University（卡内基梅隆大学）
We investigate the representation power of graph neural networks in the semi-supervised node classification task under heterophily or low homophily, i.e., in networks where connected nodes may have different class labels and dissimilar features. Many popular GNNs fail to generalize to this setting, and are even outperformed by models that ignore the graph structure (e.g., multilayer perceptrons). Motivated by this limitation, we identify a set of key designs -- ego- and neighbor-embedding separation, higher-order neighborhoods, and combination of intermediate representations -- that boost lear…
Graph Agent Network: Empowering Nodes with Inference Capabilities for Adversarial Resilience
Sichuan University（四川大学）；Chengdu University of Information Technology（成都信息工程大学）；Shihezi University（石河子大学）；Tianjin University（天津大学）；Huazhong University of Science and Technology（华中科技大学）
Technische Universität München（慕尼黑工业大学）；Mercateo（Mercateo公司）；fortiss GmbH（fortiss有限公司）；Ludwig-Maximilians-University（慕尼黑大学）
In graph neural networks (GNNs), message passing iteratively aggregates nodes' information from their direct neighbors while neglecting the sequential nature of multi-hop node connections. Such sequential node connections e.g., metapaths, capture critical insights for downstream tasks. Concretely, in recommender systems (RSs), disregarding these insights leads to inadequate distillation of collaborative signals. In this paper, we employ collaborative subgraphs (CSGs) and metapaths to form metapath-aware subgraphs, which explicitly capture sequential semantics in graph structures. We propose me…
The attention mechanism enables graph neural networks (GNNs) to learn the attention weights between the target node and its one-hop neighbors, thereby improving the performance further. However, most existing GNNs are oriented toward homogeneous graphs, and in which each layer can only aggregate the information of one-hop neighbors. Stacking multilayer networks introduces considerable noise and easily leads to over smoothing. We propose here a multihop heterogeneous neighborhood information fusion graph representation learning method (MHNF). Specifically, we propose a hybrid metapath autonomou…
Recently, the Weisfeiler-Lehman (WL) graph isomorphism test was used to measure the expressive power of graph neural networks (GNN). It was shown that the popular message passing GNN cannot distinguish between graphs that are indistinguishable by the 1-WL test (Morris et al. 2018; Xu et al. 2019). Unfortunately, many simple instances of graphs are indistinguishable by the 1-WL test. In search for more expressive graph learning models we build upon the recent k-order invariant and equivariant graph neural networks (Maron et al. 2019a,b) and present two results: First, we show that such k-order …
Graph4BiLO: Graph Neural Network Approximation for Bilevel Mixed-Integer Linear Optimization
Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida（南佛罗里达大学贝利尼人工智能、网络安全与计算学院）；Muma College of Business, University of South Florida（南佛罗里达大学穆玛商学院）；University of South Florida（南佛罗里达大学）
Accurate traffic forecasting, the foundation of intelligent transportation systems (ITS), has never been more significant than nowadays due to the prosperity of smart cities and urban computing. Recently, Graph Neural Network truly outperforms the traditional methods. Nevertheless, the most conventional GNN-based model works well while given a pre-defined graph structure. And the existing methods of defining the graph structures focus purely on spatial dependencies and ignore the temporal correlation. Besides, the semantics of the static pre-defined graph adjacency applied during the whole tra…
Kolmogorov Arnold networks (KAN) for aerodynamic prediction: a comparison with MLPs and GNNs
ETSIAE-UPM-School of Aeronautics, Universidad Politécnica de Madrid（马德里理工大学航空学院）；Center for Computational Simulation, Universidad Politécnica de Madrid（马德里理工大学计算模拟中心）；Institute for Cross-Disciplinary Physics and Complex Systems (IFISC, CSIC-UIB)（跨学科物理与复杂系统研究所（IFISC，CSIC-UIB））
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息系统学院）；JD Explore Academy（京东探索研究院）；School of Computing, Macquarie University（麦考瑞大学计算学院）
Advanced Financial Fraud Detection Using GNN-CL Model
Columbia University（哥伦比亚大学）；Rutgers University（罗格斯大学）；College of William & Mary（威廉与玛丽学院）；Stevens Institute of Technological（史蒂文斯理工学院）；University of California, Davis（加利福尼亚大学戴维斯分校）
A key challenge in metabolomics is annotating measured spectra from a biological sample with chemical identities. Currently, only a small fraction of measurements can be assigned identities. Two complementary computational approaches have emerged to address the annotation problem: mapping candidate molecules to spectra, and mapping query spectra to molecular candidates. In essence, the candidate molecule with the spectrum that best explains the query spectrum is recommended as the target molecule. Despite candidate ranking being fundamental in both approaches, no prior works utilized rank lear…
Graph Neural Network (GNN) has been demonstrated its effectiveness in dealing with non-Euclidean structural data. Both spatial-based and spectral-based GNNs are relying on adjacency matrix to guide message passing among neighbors during feature aggregation. Recent works have mainly focused on powerful message passing modules, however, in this paper, we show that none of the message passing modules is necessary. Instead, we propose a pure multilayer-perceptron-based framework, Graph-MLP with the supervision signal leveraging graph structure, which is sufficient for learning discriminative node …
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Trainable Adaptive Activation Function Structure (TAAFS) Enhances Neural Network Force Field Performance with Only Dozens of Additional Parameters
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所 处理器国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Aggregation-aware MLP: An Unsupervised Approach for Graph Message-passing
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Michigan State University（密歇根州立大学）；Alibaba Group（阿里巴巴集团）
Dept. of Signal Theory and Communications, Rey Juan Carlos University, Madrid, Spain（信号理论与通信系，雷亚尔·卡洛斯大学，马德里，西班牙）；Dept. of Electrical and Computer Engineering, University of Rochester（电气与计算机工程系，罗切斯特大学）
A Survey on Deep Neural Networks in Collaborative Filtering Recommendation Systems
Center for Artificial Intelligence and Technology（人工智能与技术中心）；Faculty of Information Science and Technology（信息科学与技术学院）；The National University of Malaysia（马来西亚国立大学）
