异构体系结构上不规则点对点通信的节点感知策略性能表征
Characterizing the Performance of Node-Aware Strategies for Irregular Point-to-Point Communication on Heterogeneous Architectures
摘要:
超级计算机体系结构由于包含异构计算节点而趋向于更高的计算吞吐量。这些多GPU节点提高了节点上的计算效率,同时也增加了要传输的数据量和潜在数据流路径的数量。在这项工作中,我们通过性能建模的异构计算环境中的MPI不规则的点对点通信的性能特征,展示了设备感知和分级通过主机通信技术的标准通信策略的局限性。提出的模型建议通过主机进程,然后使用节点感知的通信策略,高节点间的消息计数进行通信数据。值得注意的是,这些模型还预测,当与大量节点进行通信时,利用所有可用的CPU内核来通信节点间数据的节点感知通信会导致最高性能的策略。模型验证是通过分布式稀疏矩阵向量产品的不规则点对点通信模式的案例研究。重要的是,我们包括一个讨论的影响模型预测的通信策略设计新兴的超级计算机架构。
英文摘要:
Supercomputer architectures are trending toward higher computational throughput due to the inclusion of heterogeneous compute nodes. These multi-GPU nodes increase on-node computational efficiency, while also increasing the amount of data to be communicated and the number of potential data flow paths. In this work, we characterize the performance of irregular point-to-point communication with MPI on heterogeneous compute environments through performance modeling, demonstrating the limitations of standard communication strategies for both device-aware and staging-through-host communication techniques. Presented models suggest staging communicated data through host processes then using node-aware communication strategies for high inter-node message counts. Notably, the models also predict that node-aware communication utilizing all available CPU cores to communicate inter-node data leads to the most performant strategy when communicating with a high number of nodes. Model validation is provided via a case study of irregular point-to-point communication patterns in distributed sparse matrix-vector products. Importantly, we include a discussion on the implications model predictions have on communication strategy design for emerging supercomputer architectures.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。