arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.15348cs.LGcs.DC
收藏

ReInc:动态图神经网络的规模化训练

ReInc: Scaling Training of Dynamic Graph Neural Networks

Georgia Institute of Technology(佐治亚理工学院)

Mingyu Guan, Saumia Singhal, Taesoo Kim, Anand Padmanabha Iyer

AI总结:

本文提出ReInc系统,通过重用中间结果、增量计算聚合、两级缓存机制和新型分布式训练策略,解决了动态图神经网络的大规模高效训练问题,实现最高一个数量级加速。

摘要:

动态图神经网络(DGNNs)因其在交通网络预测、流行病预测和社交网络分析等多个领域的适用性而受到广泛关注。本文提出了ReInc,这是一个旨在实现大规模图上DGNNs高效且可扩展训练的系统。ReInc引入了关键创新,充分利用了DGNNs中固有的图神经网络(GNNs)与循环神经网络(RNNs)的独特组合。通过重用中间结果并跨连续图快照增量计算聚合,ReInc显著提升了计算效率。为支持这些优化,ReInc采用了一种新型两级缓存机制,并配备了与DGNN执行流程相匹配的专用缓存策略。此外,ReInc通过一种新的分布式训练策略,解决了动态图中管理结构依赖和时间依赖的挑战。该方法消除了与访问远程特征和重新分配中间结果相关的通信开销。实验结果表明,与最先进的框架相比,ReInc实现了高达一个数量级的加速,并在多种动态GNN架构和真实图数据集上进行了测试。

英文摘要:

Dynamic Graph Neural Networks (DGNNs) have gained widespread attention due to their applicability in diverse domains such as traffic network prediction, epidemiological forecasting, and social network analysis. In this paper, we present ReInc, a system designed to enable efficient and scalable training of DGNNs on large-scale graphs. ReInc introduces key innovations that capitalize on the unique combination of Graph Neural Networks (GNNs) and Recurrent Neural Networks (RNNs) inherent in DGNNs. By reusing intermediate results and incrementally computing aggregations across consecutive graph snapshots, ReInc significantly enhances computational efficiency. To support these optimizations, ReInc incorporates a novel two-level caching mechanism with a specialized caching policy aligned to the DGNN execution workflow. Additionally, ReInc addresses the challenges of managing structural and temporal dependencies in dynamic graphs through a new distributed training strategy. This approach eliminates communication overheads associated with accessing remote features and redistributing intermediate results. Experimental results demonstrate that ReInc achieves up to an order of magnitude speedup compared to state-of-the-art frameworks, tested across various dynamic GNN architectures and real-world graph datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑