arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.06385cs.LGcs.DC
收藏

DistGNN-MB:基于小批量抽样的x86分布式大规模图神经网络训练

DistGNN-MB: Distributed Large-Scale Graph Neural Network Training on x86 via Minibatch Sampling

Intel Corporation(英特尔公司)

Md Vasimuddin, Ramanarayan Mohanty, Sanchit Misra, Sasikanth Avancha

摘要:

在包含数十亿个顶点和边的图上,使用小批量采样大规模训练图神经网络提出了一个关键挑战:强缩放图和训练样本会导致计算量降低、通信量增加和潜在的性能损失。DistGNN-MB采用了一种新颖的历史嵌入缓存与计算通信重叠相结合,以解决这一挑战。在$3^{rd}$代英特尔至强可扩展处理器(每个插槽36个内核)的32节点(64插槽)群集上,DistGNN-MB在OGBN-Papers 100 M上训练3层GraphSAGE和GAT模型,使其在32个计算节点上分别以2秒和4.9秒的历元时间收敛。在这个规模下,DistGNN-MB训练GraphSAGE的速度比广泛使用的DistDGL快5.2倍。随着计算节点从2个扩展到32个,DistGNN-MB训练GraphSAGE和GAT的速度分别提高了10倍和17.2倍。

英文摘要:

Training Graph Neural Networks, on graphs containing billions of vertices and edges, at scale using minibatch sampling poses a key challenge: strong-scaling graphs and training examples results in lower compute and higher communication volume and potential performance loss. DistGNN-MB employs a novel Historical Embedding Cache combined with compute-communication overlap to address this challenge. On a 32-node (64-socket) cluster of $3^{rd}$ generation Intel Xeon Scalable Processors with 36 cores per socket, DistGNN-MB trains 3-layer GraphSAGE and GAT models on OGBN-Papers100M to convergence with epoch times of 2 seconds and 4.9 seconds, respectively, on 32 compute nodes. At this scale, DistGNN-MB trains GraphSAGE 5.2x faster than the widely-used DistDGL. DistGNN-MB trains GraphSAGE and GAT 10x and 17.2x faster, respectively, as compute nodes scale from 2 to 32.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑