稀疏图神经网络在密集硬件上的快速训练
Fast Training of Sparse Graph Neural Networks on Dense Hardware
摘要:
图神经网络由于其自然编码关系输入数据的能力以及通过对图邻接矩阵的稀疏表示进行操作而扩展到大型图的能力,近年来变得越来越流行。当我们希望使用自定义硬件扩展这些模型时,一个自然的假设是我们需要针对稀疏操作和/或动态控制流定制的硬件。在这项工作中,我们通过使用针对固定大小数据的密集计算的平台来扩展稀疏图神经网络来质疑这一假设。从稀疏矩阵上的数值算法优化中汲取灵感,我们开发了一些技术,可以使用512核TPUv 2 Pod在13分钟内训练Allamanis等人[2018]的稀疏图神经网络模型,而原始训练几乎需要一天。
英文摘要:
Graph neural networks have become increasingly popular in recent years due to their ability to naturally encode relational input data and their ability to scale to large graphs by operating on a sparse representation of graph adjacency matrices. As we look to scale up these models using custom hardware, a natural assumption would be that we need hardware tailored to sparse operations and/or dynamic control flow. In this work, we question this assumption by scaling up sparse graph neural networks using a platform targeted at dense computation on fixed-size data. Drawing inspiration from optimization of numerical algorithms on sparse matrices, we develop techniques that enable training the sparse graph neural network model from Allamanis et al. [2018] in 13 minutes using a 512-core TPUv2 Pod, whereas the original training takes almost a day.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。