TEMPI：一个具有CUDA感知数据库规范表示的插入式MPI库
TEMPI: An Interposed MPI Library with a Canonical Representation of CUDA-aware Datatypes
摘要：
MPI派生的数据库是一种抽象，它简化了MPI应用程序中非连续数据的处理。这些数据库是在运行时从MPI标准中定义的命名类型原语递归构造的。最近，CUDA感知MPI实现的开发和部署鼓励了分布式高性能MPI代码向使用GPU的过渡。这样的实现允许MPI函数直接在GPU缓冲区上操作，从而简化了GPU计算到MPI代码中的集成。这项工作首先提出了一种新的数据类型处理策略，嵌套跨越数据库，找到了一个中间地带之间的专门或通用的处理在以前的工作。这项工作还表明，非连续数据处理的性能特征可以用经验系统测量来建模，并用于透明地改善MPI_Send/Recv延迟。最后，尽管大量关注非连续GPU数据和CUDA感知的MPI实现，但不能认为良好的性能是理所当然的。这项工作通过MPI插入器库TEMPI展示了其贡献。TEMPI可以与现有的MPI部署一起使用，而无需更改系统或应用程序。最终，这项工作的插入库模型表明，MPI_Pack的加速高达242000倍，MPI_Send的加速高达59000倍相比，MPI的实施部署在一个领导级的超级计算机。这在具有3072个进程的3D晕圈交换中产生超过917倍的加速。
英文摘要：
MPI derived datatypes are an abstraction that simplifies handling of non-contiguous data in MPI applications. These datatypes are recursively constructed at runtime from primitive Named Types defined in the MPI standard. More recently, the development and deployment of CUDA-aware MPI implementations has encouraged the transition of distributed high-performance MPI codes to use GPUs. Such implementations allow MPI functions to directly operate on GPU buffers, easing integration of GPU compute into MPI codes. This work first presents a novel datatype handling strategy for nested strided datatypes, which finds a middle ground between the specialized or generic handling in prior work. This work also shows that the performance characteristics of non-contiguous data handling can be modeled with empirical system measurements, and used to transparently improve MPI_Send/Recv latency. Finally, despite substantial attention to non-contiguous GPU data and CUDA-aware MPI implementations, good performance cannot be taken for granted. This work demonstrates its contributions through an MPI interposer library, TEMPI. TEMPI can be used with existing MPI deployments without system or application changes. Ultimately, the interposed-library model of this work demonstrates MPI_Pack speedup of up to 242000x and MPI_Send speedup of up to 59000x compared to the MPI implementation deployed on a leadership-class supercomputer. This yields speedup of more than 917x in a 3D halo exchange with 3072 processes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
