FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
Institute of Automation, CAS(中国科学院自动化研究所);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院);Shanghai Jiao Tong University(上海交通大学);AiRiA
Graph Neural Networks (GNNs) have shown great superiority on non-Euclidean graph data, achieving ground-breaking performance on various graph-related tasks. As a practical solution to train GNN on large graphs with billions of nodes and edges, the sampling-based training is widely adopted by existing training frameworks. However, through an in-depth analysis, we observe that the efficiency of existing sampling-based training frameworks is still limited due to the key bottlenecks lying in all three phases of sampling-based training, i.e., subgraph sample, memory IO, and computation. To this end, we propose FastGL, a GPU-efficient Framework for accelerating sampling-based training of GNN at Large scale by simultaneously optimizing all above three phases, taking into account both GPU characteristics and graph structure. Specifically, by exploiting the inherent overlap within graph structures, FastGL develops the Match-Reorder strategy to reduce the data traffic, which accelerates the memory IO without incurring any GPU memory overhead. Additionally, FastGL leverages a Memory-Aware computation method, harnessing the GPU memory's hierarchical nature to mitigate irregular data access during computation. FastGL further incorporates the Fused-Map approach aimed at diminishing the synchronization overhead during sampling. Extensive experiments demonstrate that FastGL can achieve an average speedup of 11.8x, 2.2x and 1.5x over the state-of-the-art frameworks PyG, DGL, and GNNLab, respectively.Our code is available at https://github.com/a1bc2def6g/fastgl-ae.
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training
Princeton University(普林斯顿大学);Zhongguancun Haihua Institute for Frontier Information Technology(中关村海华前沿信息技术研究院);Peking University(北京大学);Pazhou Lab(琶洲实验室);Massachusetts Institute of Technology(麻省理工学院);Microsoft Research(微软研究院)
Accelerating Scalable Graph Neural Network Inference with Node-Adaptive Propagation
The University of Queensland(昆士兰大学);Peking University(北京大学);National Engineering Laboratory for Big Data Analysis and Applications(国家大数据分析与应用工程实验室);Beijing University of Posts and Telecommunications(北京邮电大学);Griffith University(格里菲斯大学)
Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classification
The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of California, Merced(加州大学默塞德分校);University of California, Los Angeles(加州大学洛杉矶分校);Createlink Technology(创联科技);School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区网络空间安全学院)
RoSGAS: Adaptive Social Bot Detection with Reinforced Self-Supervised GNN Architecture Search
University of Science and Technology of China(中国科学技术大学);Key Laboratory of Cyberculture Content Cognition and Detection, Ministry of Culture and Tourism(文化和旅游部网络文化内容认知与检测重点实验室);University of Leeds(利兹大学);National Engineering Laboratory for Public Safety Risk Perception and Control by Big Data(公共安全风险防控大数据国家工程实验室);Beihang University(北京航空航天大学)
NeuraLSP: A Neural Spectral Preconditioner for Accelerating PDE Solvers
Department of Applied Mathematics and Statistics, Stony Brook University(应用数学与统计学系,史泰尼斯布鲁克大学);Department of Computer Science, Stony Brook University(计算机科学系,史泰尼斯布鲁克大学)
RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Corrections
Institute for Interdisciplinary Computational Sciences, Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Be'er Sheva, Israel(交叉学科计算科学研究所,计算机与信息科学学院,内盖夫本·古里安大学,以色列贝尔谢瓦);Department of Mathematics and Computer Science, Emory University, Atlanta, GA, USA(数学与计算机科学系,埃默里大学,美国亚特兰大,GA)
VS-Graph: Scalable and Efficient Graph Classification Using Hyperdimensional Computing
Department of Electrical and Computer Engineering, George Mason University, Fairfax, VA, USA(电气与计算机工程系,乔治·马歇尔大学,弗吉尼亚州 Fairfax);Oak Ridge National Laboratory, Oak Ridge, TN, USA(橡树岭国家实验室,橡树岭,田纳西州)
Domain-Decomposed Graph Neural Network Surrogate Modeling for Ice Sheets
Institute for Computational and Mathematical Engineering, Stanford University(计算与数学工程研究所,斯坦福大学);Department of Scientific Machine Learning, Sandia National Laboratories(科学机器学习系,桑迪亚国家实验室);Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory(先进计算、数学和数据 division,太平洋西北国家实验室);Delft Institute of Applied Mathematics, Delft University of Technology(应用数学研究所,代尔夫特理工大学);Department of Energy Science Engineering, Stanford University(能源科学与工程系,斯坦福大学)
GDEGAN: Gaussian Dynamic Equivariant Graph Attention Network for Ligand Binding Site Prediction
Department of Artificial Intelligence, Indian Institute of Technology, Kharagpur, India(人工智能系,印度理工学院,Khargpur,印度);Department of Computer Science, Indian Institute of Technology, Kharagpur, India(计算机科学系,印度理工学院,Khargpur,印度)
ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphs
Department of Computer Science, Rice University(计算机科学系,里士大学);Rice Advanced Materials Institute, Rice University(里士大学先进材料研究所);Department of Materials Science and Nanoengineering, Rice University(材料科学与纳米工程系,里士大学);Thayer School of Engineering, Dartmouth College(达特茅斯学院泰勒工程学院);Department of Computer Sciences, University of Wisconsin–Madison(威斯康星大学麦迪逊分校计算机科学系)
MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization
Georgia Institute of Technology(佐治亚理工学院);Analytics Center of Excellence, IQVIA(IQVIA卓越分析中心);North Carolina State University(北卡罗来纳州立大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)