Ginex：通过可证明的最佳内存缓存在单机上进行支持SSD的十亿级图神经网络训练
Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Caching
摘要：
最近，图神经网络（GNN）作为一种强大的工具，可以有效地为图结构数据的各种推理任务提供服务，因此受到了人们的关注。随着真实世界图形的规模不断扩大，GNN训练系统面临着可扩展性的挑战。分布式训练是通过扩展CPU节点来解决这一挑战的流行方法。然而，基于磁盘的GNN训练并没有受到太多关注，它可以通过利用NVMe SSD等高性能存储设备以更经济高效的方式扩展单节点系统。我们观察到，主存储器和磁盘之间的数据移动是基于SSD的训练系统的主要瓶颈，而传统的GNN训练管道在不考虑这种开销的情况下是次优的。因此，我们提出了Ginex，这是第一个基于SSD的GNN训练系统，可以在一台机器上处理数十亿规模的图数据集。受编译器优化中的检查器-执行器执行模型的启发，Ginex通过分离采样和收集阶段来重构GNN训练管道。这种分离使Ginex能够实现一种可证明的最佳替换算法，称为Belady算法，用于在内存中缓存特征向量，这些特征向量占I/O访问的主要部分。根据我们对40亿级图数据集的评估，Ginex的训练吞吐量平均比SSD扩展的PyTorch Geometric高2.11倍（最高可达2.67倍）。
英文摘要：
Recently, Graph Neural Networks (GNNs) have been receiving a spotlight as a powerful tool that can effectively serve various inference tasks on graph structured data. As the size of real-world graphs continues to scale, the GNN training system faces a scalability challenge. Distributed training is a popular approach to address this challenge by scaling out CPU nodes. However, not much attention has been paid to disk-based GNN training, which can scale up the single-node system in a more cost-effective manner by leveraging high-performance storage devices like NVMe SSDs. We observe that the data movement between the main memory and the disk is the primary bottleneck in the SSD-based training system, and that the conventional GNN training pipeline is sub-optimal without taking this overhead into account. Thus, we propose Ginex, the first SSD-based GNN training system that can process billion-scale graph datasets on a single machine. Inspired by the inspector-executor execution model in compiler optimization, Ginex restructures the GNN training pipeline by separating sample and gather stages. This separation enables Ginex to realize a provably optimal replacement algorithm, known as Belady's algorithm, for caching feature vectors in memory, which account for the dominant portion of I/O accesses. According to our evaluation with four billion-scale graph datasets, Ginex achieves 2.11x higher training throughput on average (up to 2.67x at maximum) than the SSD-extended PyTorch Geometric.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
