arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.14939cs.LGcs.ARcs.DC
收藏

FastGL:一种用于大规模加速基于采样的GNN训练的GPU高效框架

FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale

Institute of Automation, CAS(中国科学院自动化研究所);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院);Shanghai Jiao Tong University(上海交通大学);AiRiA

Zeyu Zhu, Peisong Wang, Qinghao Hu, Gang Li, Xiaoyao Liang, Jian Cheng

AI总结:

针对大规模图神经网络基于采样的训练在子图采样、内存IO和计算三阶段的瓶颈,提出GPU高效框架FastGL,通过Match-Reorder、Memory-Aware和Fused-Map方法优化全流程,较PyG、DGL和GNNLab实现显著加速。

摘要:

图神经网络(GNNs)在非欧几里得图数据上展现出极大优势,在各种图相关任务中取得了突破性性能。作为在包含数十亿节点和边的图上训练GNN的实用解决方案,基于采样的训练被现有训练框架广泛采用。然而,通过深入分析,我们观察到现有基于采样的训练框架的效率仍然受限,这主要是由于基于采样的训练的三个阶段(即子图采样、内存IO和计算)均存在关键瓶颈。为此,我们提出FastGL,这是一种GPU高效的框架,通过同时优化上述三个阶段,兼顾GPU特性和图结构,实现大规模加速GNN基于采样的训练。具体而言,通过利用图结构固有的重叠性,FastGL开发了Match-Reorder策略以减少数据流量,在不产生任何GPU内存开销的情况下加速内存IO。此外,FastGL利用Memory-Aware计算方法,发挥GPU内存的层次特性,以缓解计算期间的不规则数据访问。FastGL进一步引入Fused-Map方法,旨在减少采样期间的同步开销。大量实验表明,FastGL相较于最先进的框架PyG、DGL和GNNLab,分别实现了平均11.8倍、2.2倍和1.5倍的加速。代码可在https://github.com/a1bc2def6g/fastgl-ae获取。

英文摘要:

Graph Neural Networks (GNNs) have shown great superiority on non-Euclidean graph data, achieving ground-breaking performance on various graph-related tasks. As a practical solution to train GNN on large graphs with billions of nodes and edges, the sampling-based training is widely adopted by existing training frameworks. However, through an in-depth analysis, we observe that the efficiency of existing sampling-based training frameworks is still limited due to the key bottlenecks lying in all three phases of sampling-based training, i.e., subgraph sample, memory IO, and computation. To this end, we propose FastGL, a GPU-efficient Framework for accelerating sampling-based training of GNN at Large scale by simultaneously optimizing all above three phases, taking into account both GPU characteristics and graph structure. Specifically, by exploiting the inherent overlap within graph structures, FastGL develops the Match-Reorder strategy to reduce the data traffic, which accelerates the memory IO without incurring any GPU memory overhead. Additionally, FastGL leverages a Memory-Aware computation method, harnessing the GPU memory's hierarchical nature to mitigate irregular data access during computation. FastGL further incorporates the Fused-Map approach aimed at diminishing the synchronization overhead during sampling. Extensive experiments demonstrate that FastGL can achieve an average speedup of 11.8x, 2.2x and 1.5x over the state-of-the-art frameworks PyG, DGL, and GNNLab, respectively.Our code is available at https://github.com/a1bc2def6g/fastgl-ae.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑