Hector：一种在 GPU 架构中实现关系图神经网络的高效编程与编译框架
Hector: An Efficient Programming and Compilation Framework for Implementing Relational Graph Neural Networks in GPU Architectures
AI总结：
本文提出 Hector 两级中间表示与代码生成框架，通过减少内存访问、灵活数据访问及解耦优化，在 GPU 上高效实现 RGNN，并显著降低编程成本。
摘要：
关系图神经网络（RGNN）是一类具有专用结构的图神经网络，用于对异构图中不同类型的节点和边进行建模。尽管 RGNN 因其多功能性和准确性而在许多现实应用中得到越来越多的采用，但它们也带来了性能和系统设计方面的挑战：固有的内存密集型计算模式、编程接口与内核 API 之间的差距，以及内核优化因与数据布局和异构性耦合而产生的繁重编程工作。为系统地解决这些挑战，我们提出了 Hector，这是一种新颖的两级中间表示及其代码生成框架，它：(a) 捕获 RGNN 模型的关键属性，以及在算子间调度和物化过程中减少内存访问的机会；(b) 生成具有灵活数据访问方案的代码，以消除冗余数据拷贝；(c) 将模型语义、数据布局和算子特定优化彼此解耦，以减少编程工作量。通过基于一个通用矩阵乘法（GEMM）模板和一个节点/边遍历模板构建，在运行 Deep Graph Library（DGL）和 Open Graph Benchmark（OGB）提供的异构图时，与选定模型（即 RGCN、RGAT 和 HGT）上最先进的公开系统相比，Hector 在推理中实现了最高 9.9 倍的加速，在训练中实现了最高 43.7 倍的加速。此外，Hector 在这些测试中没有触发任何内存不足（OOM）异常。我们还提出了线性算子重排和紧凑物化，以进一步将系统加速最高 3.8 倍。作为编程工作量减少的一个指标，Hector 用 51 行代码表达这三个模型，并生成总计 8K 行 CUDA 和 C++ 代码。
英文摘要：
Relational graph neural networks (RGNNs) are graph neural networks with dedicated structures for modeling the different types of nodes and edges in heterogeneous graphs. While RGNNs have been increasingly adopted in many real-world applications due to their versatility and accuracy, they pose performance and system design challenges: inherent memory-intensive computation patterns, the gap between the programming interface and kernel APIs, and heavy programming effort in optimizing kernels caused by their coupling with data layout and heterogeneity. To systematically address these challenges, we propose Hector, a novel two-level intermediate representation and its code generator framework, that (a) captures the key properties of RGNN models, and opportunities to reduce memory accesses in inter-operator scheduling and materialization, (b) generates code with flexible data access scheme to eliminate redundant data copies, (c) decouples model semantics, data layout, and operators-specific optimization from each other to reduce programming effort. By building on one general matrix multiply (GEMM) template and a node/edge traversal template, Hector achieves up to 9.9x speed-up in inference and 43.7x speed-up in training compared with the state-of-the-art public systems on select models, i.e., RGCN, RGAT and HGT, when running heterogeneous graphs provided by Deep Graph Library (DGL) and Open Graph Benchmark (OGB). In addition, Hector does not trigger any out-of-memory (OOM) exception in these tests. We also propose the linear operator reorder and compact materialization to further accelerate the system by up to 3.8x. As an indicator of programming effort reduction, Hector takes in 51 lines of code expressing the three models and generates a total of 8K lines of CUDA and C++ code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
