CUDA + MPI设计规则的机器学习
Machine Learning for CUDA+MPI Design Rules
摘要：
我们提出了一种新的策略，自动探索关键CUDA+MPI程序的设计空间，并提供区分快速实现慢的设计规则。在这样的程序中，操作的顺序（例如，GPU内核、MPI通信）和将操作分配给资源（例如，GPU流）使得可能的设计空间巨大。系统专家的任务是重新设计和重新优化这些程序，以有效地利用每个新平台。这项工作提供了一个原型工具，以减少这一负担。在我们的方法中，CUDA和MPI操作的有向无环图定义了程序的设计空间。蒙特卡罗树搜索发现设计空间中对程序性能有很大影响的区域。序列到向量的转换为每个探索的实现定义了特征，并且每个实现根据其相对性能被分配了一个类标签。决策树在特征和标签上进行训练，为每个类生成设计规则;系统专家可以使用这些规则来指导其实现。我们展示了我们的策略，使用一个关键的内核从科学计算-稀疏矩阵向量乘法-在一个平台上与多个MPI行列和GPU流。
英文摘要：
We present a new strategy for automatically exploring the design space of key CUDA+MPI programs and providing design rules that discriminate slow from fast implementations. In such programs, the order of operations (e.g., GPU kernels, MPI communication) and assignment of operations to resources (e.g., GPU streams) makes the space of possible designs enormous. Systems experts have the task of redesigning and reoptimizing these programs to effectively utilize each new platform. This work provides a prototype tool to reduce that burden. In our approach, a directed acyclic graph of CUDA and MPI operations defines the design space for the program. Monte-Carlo tree search discovers regions of the design space that have large impact on the program's performance. A sequence-to-vector transformation defines features for each explored implementation, and each implementation is assigned a class label according to its relative performance. A decision tree is trained on the features and labels to produce design rules for each class; these rules can be used by systems experts to guide their implementations. We demonstrate our strategy using a key kernel from scientific computing -- sparse-matrix vector multiplication -- on a platform with multiple MPI ranks and GPU streams.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
