AWS Shanghai AI Lab(AWS上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Southern University of Science and Technology(南方科技大学);TensorChord;The George Washington University(乔治华盛顿大学)
Peiqi Yin, Xiao Yan, Jinjing Zhou, Qiang Fu, Zhenkun Cai, James Cheng, Bo Tang, Minjie Wang
While many systems have been developed to train Graph Neural Networks (GNNs), efficient model inference and evaluation remain to be addressed. For instance, using the widely adopted node-wise approach, model evaluation can account for up to 94% of the time in the end-to-end training process due to neighbor explosion, which means that a node accesses its multi-hop neighbors. On the other hand, layer-wise inference avoids the neighbor explosion problem by conducting inference layer by layer such that the nodes only need their one-hop neighbors in each layer. However, implementing layer-wise inference requires substantial engineering efforts because users need to manually decompose a GNN model into layers for computation and split workload into batches to fit into device memory. In this paper, we develop Deep Graph Inference (DGI) -- a system for easy and efficient GNN model inference, which automatically translates the training code of a GNN model for layer-wise execution. DGI is general for various GNN models and different kinds of inference requests, and supports out-of-core execution on large graphs that cannot fit in CPU memory. Experimental results show that DGI consistently outperforms layer-wise inference across different datasets and hardware settings, and the speedup can be over 1,000x.
Graph Neural Networks with Learnable Structural and Positional Representations
Nanyang Technological University(南洋理工大学);Loyola Marymount University(洛约拉马利蒙特大学);Mila(米拉);University of Montréal(蒙特利尔大学);CIFAR(加拿大高级研究所);National University of Singapore(新加坡国立大学)
Department of Computer Science, Control and Management Engineering, Sapienza University of Rome, Rome, Italy(计算机科学与控制管理工程系,罗马萨皮恩扎大学,意大利罗马);Institute of Information Science and Technologies "Alessandro Faedo" - ISTI-CNR, Pisa, Italy(信息科学与技术研究所 "Alessandro Faedo" - ISTI-CNR,意大利比萨);Information Engineering Department, University of Pisa, Pisa, Italy(信息工程系,比萨大学,意大利比萨);CNRS, i3S, Inria(国家科学研究中心 CNRS、i3S、Inria)
Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs
Fuzhou University(福州大学);HKUST (Guangzhou)(香港科技大学(广州));Shandong Provincial Key Laboratory of Computer Networks(山东省计算机网络重点实验室);Shandong Fundamental Research Center for Computer Science(山东计算机科学基础研究中心)
Out-of-Distribution Generalization on Graphs via Progressive Inference
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Shaanxi Provincial Key Laboratory of Big Data Knowledge Engineering, Xi’an Jiaotong University(西安交通大学陕西省大数据知识工程重点实验室);School of Distance Education, Xi’an Jiaotong University(西安交通大学远程教育学院);University of Virginia(弗吉尼亚大学)
Nanyang Technological University(南洋理工大学);University of Cambridge(剑桥大学);Loyola Marymount University(洛约拉马利蒙特大学);Mila(米拉人工智能研究所);University of Montréal(蒙特利尔大学);National University of Singapore(新加坡国立大学)
Learning How Much to Think: Difficulty-Aware Dynamic MoEs for Graph Node Classification
Institute of Cyberspace Security, Zhejiang University of Technology(浙江工业大学网络空间安全研究院);Binjiang Cyberspace Security Institute of ZJUT(浙江工业大学滨江网络空间安全研究院);Soovar Technologies Co., Ltd.(Soovar科技有限公司);Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology(香港科技大学电子及计算机工程学系)