arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.00352cs.LG
收藏

优化图神经网络稀疏矩阵乘法

Optimizing Sparse Matrix Multiplications for Graph Neural Networks

University of Leeds(利兹大学);Alibaba Group(阿里巴巴集团)

Shenghao Qiu, You Liang, Zheng Wang

摘要:

图神经网络(GNN)正在成为一种强大的图结构建模技术。由于真实世界图数据的稀疏性,GNN的性能受到计算中涉及的大量稀疏矩阵乘法(SpMM)操作的限制。虽然正确的稀疏矩阵存储格式因输入数据而异,但现有的深度学习框架采用单一的静态存储格式,这留下了很大的改进空间。本文研究了稀疏矩阵存储格式的选择如何影响GNN的性能。我们观察到,选择合适的稀疏矩阵存储格式可以显着提高GNN的训练性能,但正确的格式取决于输入工作负载,并且可以随着GNN在输入图上的迭代而变化。然后,我们开发了一个预测模型,根据输入矩阵动态选择GNN层使用的稀疏矩阵存储格式。我们的模型首先使用训练矩阵样本进行离线训练,经过训练的模型可以应用于任何输入矩阵和具有SpMM计算的GNN内核。我们在PyTorch之上实现了我们的方法,并将其应用于使用真实和合成数据集在多核CPU上运行的5个代表性GNN模型。实验结果表明,我们的方法为GNN运行时间提供了1.17倍(高达3倍)的平均加速比。

英文摘要:

Graph neural networks (GNNs) are emerging as a powerful technique for modeling graph structures. Due to the sparsity of real-world graph data, GNN performance is limited by extensive sparse matrix multiplication (SpMM) operations involved in computation. While the right sparse matrix storage format varies across input data, existing deep learning frameworks employ a single, static storage format, leaving much room for improvement. This paper investigates how the choice of sparse matrix storage formats affect the GNN performance. We observe that choosing a suitable sparse matrix storage format can significantly improve the GNN training performance, but the right format depends on the input workloads and can change as the GNN iterates over the input graph. We then develop a predictive model to dynamically choose a sparse matrix storage format to be used by a GNN layer based on the input matrices. Our model is first trained offline using training matrix samples, and the trained model can be applied to any input matrix and GNN kernels with SpMM computation. We implement our approach on top of PyTorch and apply it to 5 representative GNN models running on a multi-core CPU using real-life and synthetic datasets. Experimental results show that our approach gives an average speedup of 1.17x (up to 3x) for GNN running time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑