arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.10569cs.LGcs.DC
收藏

GraphTheta:一个具有灵活训练策略的分布式图神经网络学习系统

GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy

Ant Group(蚂蚁集团);Fudan University(复旦大学);Guangzhou University(广州大学);Zhejiang University(浙江大学);Tsinghua University(清华大学)

Yongchao Liu, Houyi Li, Guowei Zhang, Xintan Zeng, Yongyong Li, Bin Huang, Peng Zhang, Zhao Li, Xiaowei Zhu, Changhua He, Wenguang Chen

摘要:

图神经网络(GNNs)已被证明是一个强大的工具,用于分析非欧几里德图数据。然而,缺乏有效的分布式图学习系统严重阻碍了GNN的应用,特别是当图很大并且GNN相对较深时。在这里,我们提出了GraphTheta,第一个分布式和可扩展的图形学习系统建立在以顶点为中心的分布式图形处理与神经网络运营商实现为用户定义的功能。该系统支持多种训练策略,并在低内存的分布式(虚拟)机器上实现高效和可扩展的大图学习。为了方便图卷积,GraphTheta提出了一种新的图学习抽象NN-TGAR,以弥合图处理和图深度学习之间的差距。提出了一种分布式图引擎来进行混合并行执行的随机梯度下降优化,并支持一种新的集群批量训练策略。我们使用几个数据集来评估GraphTheta,这些数据集的网络大小从小到大。实验结果表明,GraphTheta可以很好地扩展到1,024名员工,用于在具有14亿个节点和41亿条属性边的行业规模支付宝数据集上训练内部开发的GNN,其中每个CPU虚拟机(dockers)的内存较小(5个)。$\sim$ 12 GB)。此外,GraphTheta可以通过高达$2.02\times$的性能优于DistDGL,具有更好的可扩展性,并且GraphLearn可以高达$30.56\times$。至于模型的准确性,GraphTheta能够学习与现有框架一样好的GNN。据我们所知,这项工作提出了文献中最大的边缘属性GNN学习任务。

英文摘要:

Graph neural networks (GNNs) have been demonstrated as a powerful tool for analyzing non-Euclidean graph data. However, the lack of efficient distributed graph learning systems severely hinders applications of GNNs, especially when graphs are big and GNNs are relatively deep. Herein, we present GraphTheta, the first distributed and scalable graph learning system built upon vertex-centric distributed graph processing with neural network operators implemented as user-defined functions. This system supports multiple training strategies and enables efficient and scalable big-graph learning on distributed (virtual) machines with low memory. To facilitate graph convolutions, GraphTheta puts forward a new graph learning abstraction named NN-TGAR to bridge the gap between graph processing and graph deep learning. A distributed graph engine is proposed to conduct the stochastic gradient descent optimization with a hybrid-parallel execution, and a new cluster-batched training strategy is supported. We evaluate GraphTheta using several datasets with network sizes ranging from small-, modest- to large-scale. Experimental results show that GraphTheta can scale well to 1,024 workers for training an in-house developed GNN on an industry-scale Alipay dataset of 1.4 billion nodes and 4.1 billion attributed edges, with a cluster of CPU virtual machines (dockers) of small memory each (5$\sim$12GB). Moreover, GraphTheta can outperform DistDGL by up to $2.02\times$, with better scalability, and GraphLearn by up to $30.56\times$. As for model accuracy, GraphTheta is capable of learning as good GNNs as existing frameworks. To the best of our knowledge, this work presents the largest edge-attributed GNN learning task in the literature.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑