arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.05150cs.LGcs.SI
收藏

基于图粗化的图神经网络规模放大

Scaling Up Graph Neural Networks Via Graph Coarsening

Fudan University(复旦大学);Huawei Technologies Co. Ltd(华为技术有限公司)

Zengfeng Huang, Shengzhong Zhang, Chong Xi, Tang Liu, Min Zhou

摘要:

图神经网络的可扩展性仍然是图机器学习的主要挑战之一。由于节点的表示是通过递归地聚合和变换来自先前层的其相邻节点的表示向量来计算的,因此感受野呈指数增长,这使得标准随机优化技术无效。已经提出了各种方法来缓解这个问题,例如,基于图过滤器的预计算的基于采样的方法和技术。在本文中,我们采取了不同的方法,并提出使用图粗化来进行GNN的可扩展训练,这是通用的,非常简单,并且在训练过程中具有次线性内存和时间成本。我们提出了广泛的理论分析使用粗化操作的效果,并提供了有用的指导粗化方法的选择。有趣的是,我们的理论分析表明,粗化也可以被认为是一种正则化,并可能提高泛化能力。最后,在真实世界数据集上的实证结果表明,简单地应用现成的粗化方法,我们可以将节点数量减少多达十倍,而不会导致分类精度明显下降。

英文摘要:

Scalability of graph neural networks remains one of the major challenges in graph machine learning. Since the representation of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes from previous layers, the receptive fields grow exponentially, which makes standard stochastic optimization techniques ineffective. Various approaches have been proposed to alleviate this issue, e.g., sampling-based methods and techniques based on pre-computation of graph filters. In this paper, we take a different approach and propose to use graph coarsening for scalable training of GNNs, which is generic, extremely simple and has sublinear memory and time costs during training. We present extensive theoretical analysis on the effect of using coarsening operations and provides useful guidance on the choice of coarsening methods. Interestingly, our theoretical analysis shows that coarsening can also be considered as a type of regularization and may improve the generalization. Finally, empirical results on real world datasets show that, simply applying off-the-shelf coarsening methods, we can reduce the number of nodes by up to a factor of ten without causing a noticeable downgrade in classification accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑