arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.03671cs.DC
收藏

ARGO:面向多核处理器上可扩展图神经网络训练的自动调优运行时系统

ARGO: An Auto-Tuning Runtime System for Scalable GNN Training on Multi-Core Processor

Yi-Chien Lin, Yuyang Chen, Sameh Gobriel, Nilesh Jain, Gopi Krishna Jha, Viktor Prasanna

AI总结:

ARGO提出面向多核处理器的GNN训练运行时系统,通过多进程、核心绑定和自动调优器仅探索5%设计空间即可选择近最优配置,在112核和64核平台上分别实现最高5.06倍和4.54倍加速,并可无缝集成到DGL、PyG等主流GNN库中。

摘要:

随着图神经网络(GNN)日益普及,PyTorch-Geometric(PyG)和Deep Graph Library(DGL)等库相继被提出;这些库已成为实现GNN的事实标准,因为它们提供了面向图的API,并且专门设计用于管理图结构固有的稀疏性和不规则性。然而,这些库在多核处理器上表现出较差的可扩展性,导致可用平台资源未被充分利用并限制了性能。这是因为GNN训练是一种资源密集型工作负载,具有大量不规则数据访问,而现有库未能高效利用内存带宽。为应对这一挑战,我们提出了ARGO,这是一种用于GNN训练的新型运行时系统,可提供可扩展的性能。ARGO利用多进程和核心绑定技术来提高平台资源利用率。我们进一步开发了一个自动调优器,用于搜索多进程和核心绑定的最优配置。该自动调优器自动运行,对用户完全透明。此外,自动调优器使ARGO能够适应各种平台、GNN模型和数据集等。我们在两个平台上对两个代表性GNN模型和四个广泛使用的数据集评估了ARGO。借助所提出的自动调优器,ARGO仅探索5%的设计空间即可选择接近最优的配置。在一台拥有112核的四路Ice Lake机器和一台拥有64核的双路Sapphire Rapids机器上,ARGO分别将最先进的GNN库加速高达5.06倍和4.54倍。最后,ARGO只需少量代码即可无缝集成到广泛使用的GNN库(如DGL、PyG)中,并加速GNN训练。

英文摘要:

As Graph Neural Networks (GNNs) become popular, libraries like PyTorch-Geometric (PyG) and Deep Graph Library (DGL) are proposed; these libraries have emerged as the de facto standard for implementing GNNs because they provide graph-oriented APIs and are purposefully designed to manage the inherent sparsity and irregularity in graph structures. However, these libraries show poor scalability on multi-core processors, which under-utilizes the available platform resources and limits the performance. This is because GNN training is a resource-intensive workload with high volume of irregular data accessing, and existing libraries fail to utilize the memory bandwidth efficiently. To address this challenge, we propose ARGO, a novel runtime system for GNN training that offers scalable performance. ARGO exploits multi-processing and core-binding techniques to improve platform resource utilization. We further develop an auto-tuner that searches for the optimal configuration for multi-processing and core-binding. The auto-tuner works automatically, making it completely transparent from the user. Furthermore, the auto-tuner allows ARGO to adapt to various platforms, GNN models, datasets, etc. We evaluate ARGO on two representative GNN models and four widely-used datasets on two platforms. With the proposed autotuner, ARGO is able to select a near-optimal configuration by exploring only 5% of the design space. ARGO speeds up state-of-the-art GNN libraries by up to 5.06x and 4.54x on a four-socket Ice Lake machine with 112 cores and a two-socket Sapphire Rapids machine with 64 cores, respectively. Finally, ARGO can seamlessly integrate into widely-used GNN libraries (e.g., DGL, PyG) with few lines of code and speed up GNN training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑