arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.09398cs.DC
收藏

提升迭代应用在GPU上的性能:基于CUDA Graph的内核批处理

Boosting Performance of Iterative Applications on GPUs: Kernel Batching with CUDA Graphs

Jonah Ekelund, Stefano Markidis, Ivy Peng

AI总结:

针对GPU迭代应用中细粒度内核启动开销的瓶颈,提出基于CUDA Graph的内核批处理优化策略,通过将内核启动分组并展开为图执行,在骨架应用及多个实际求解器中实现超过1.4倍的加速。

摘要:

图形处理器(GPU)已成为在异构系统上加速科学应用的标准。然而,随着GPU速度越来越快,GPU加速应用的一个潜在性能瓶颈是启动多个细粒度内核带来的开销。CUDA Graph通过启用基于图的执行模型来解决这些性能挑战,该模型将操作捕获为节点,将依赖关系捕获为静态图中的边。从而将多个内核启动合并为一次图启动。我们提出了一种针对迭代启动内核的性能优化策略。通过将内核启动分组为迭代批次,然后将这些批次展开为CUDA Graph,迭代应用可以利用CUDA Graph来提升性能。我们通过设计一个骨架应用来分析这种方法的性能增益和开销。该骨架应用还作为将迭代求解器转换为CUDA Graph的通用示例,并用于推导性能模型。使用该骨架应用,我们展示了当为给定平台展开迭代批次时,存在一个最优的迭代批次大小,该大小与工作负载无关,在图创建的额外开销与图执行的性能增益之间取得平衡。根据工作负载的不同,我们展示了最优迭代批次大小在骨架应用中可带来超过1.4倍的加速。此外,我们展示了在Rodinia基准套件中的Hotspot和Hotspot3D以及时域有限差分(FDTD)麦克斯韦求解器中可以获得类似的加速。

英文摘要:

Graphics Processing Units (GPUs) have become the standard in accelerating scientific applications on heterogeneous systems. However, as GPUs are getting faster, one potential performance bottleneck with GPU-accelerated applications is the overhead from launching several fine-grained kernels. CUDA Graph addresses these performance challenges by enabling a graph-based execution model that captures operations as nodes and dependence as edges in a static graph. Thereby consolidating several kernel launches into one graph launch. We propose a performance optimization strategy for iteratively launched kernels. By grouping kernel launches into iteration batches and then unrolling these batches into a CUDA Graph, iterative applications can benefit from CUDA Graph for performance boosting. We analyze the performance gain and overhead from this approach by designing a skeleton application. The skeleton application also serves as a generalized example of converting an iterative solver to CUDA Graph, and for deriving a performance model. Using the skeleton application, we show that when unrolling iteration batches for a given platform, there is an optimal size of the iteration batch, which is independent of workload, balancing the extra overhead from graph creation with the performance gain of the graph execution. Depending on workload, we show that the optimal iteration batch size gives more than 1.4x speed-up in the skeleton application. Furthermore, we show that similar speed-up can be gained in Hotspot and Hotspot3D from the Rodinia benchmark suite and a Finite-Difference Time-Domain (FDTD) Maxwell solver.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑