MassiveGNN：面向大规模连接分布式图的预取高效训练方法
MassiveGNN: Efficient Training via Prefetching for Massively Connected Distributed Graphs
AI总结：
针对大规模连接分布式图上GNN训练存在的通信开销与负载不均问题，提出MassiveGNN方法，在DistDGL框架上加入参数化连续预取驱逐方案，在Perlmutter超算的OGB数据集上实现15%-40%的端到端训练性能提升。
摘要：
图神经网络（GNN）是图结构数据学习中不可或缺的工具，但其不断攀升的计算成本，尤其是在大规模连接图上的成本，给执行性能带来了严峻挑战。为解决这一问题，业界已采用分布式内存方案，例如对图进行分区以并行训练多个GNN副本。然而，需要图分区的方法通常存在通信开销和负载不均衡问题，即便采用最优分区与通信策略，邻域小批量采样的不规则性仍会导致这些问题。\n 本文在前沿的Amazon DistDGL分布式GNN框架基础上，开发了一种参数化的连续预取与驱逐方案，针对分布式图上的表示学习（采用主流GraphSAGE架构）提出了实用的权衡策略，以降低采样与通信开销。实验在美国国家能源研究科学计算中心（NERSC）的Perlmutter超级计算机上开展，针对多种OGB数据集，端到端训练性能提升约15%-40%。
英文摘要：
Graph Neural Networks (GNN) are indispensable in learning from graph-structured data, yet their rising computational costs, especially on massively connected graphs, pose significant challenges in terms of execution performance. To tackle this, distributed-memory solutions such as partitioning the graph to concurrently train multiple replicas of GNNs are in practice. However, approaches requiring a partitioned graph usually suffer from communication overhead and load imbalance, even under optimal partitioning and communication strategies due to irregularities in the neighborhood minibatch sampling. This paper proposes practical trade-offs for improving the sampling and communication overheads for representation learning on distributed graphs (using popular GraphSAGE architecture) by developing a parameterized continuous prefetch and eviction scheme on top of the state-of-the-art Amazon DistDGL distributed GNN framework, demonstrating about 15-40% improvement in end-to-end training performance on the National Energy Research Scientific Computing Center's (NERSC) Perlmutter supercomputer for various OGB datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
