并行程序中空闲波的解析建模：通信、集群拓扑和噪声影响
Analytic Modeling of Idle Waves in Parallel Programs: Communication, Cluster Topology, and Noise Impact
摘要：
HPC中的大多数分布式内存大容量同步并行程序都假设计算资源在分配的计算节点集上连续且均匀地可用。然而，单个过程的长时间一次性延迟会通过在系统中产生涟漪而引起全局扰动，即所谓的闲置波。这个过程主要由底层并行代码的通信拓扑控制。本文的研究对于理解闲置波动力学有着重要的贡献。我们研究了MPI并行程序中空闲波的传播机制。我们提出了一个有效的分析模型，其传播速度相对于通信参数和拓扑结构，特别强调稀疏的通信模式。我们研究了闲置波与MPI集体的相互作用，并表明，根据实施，集体可能是透明的波。最后，我们分析了空闲波衰减的两种机制：拓扑衰减，这是植根于系统的各部分之间的通信特性的差异，和噪声引起的衰减，这是由系统或应用程序噪声。我们表明，噪声引起的衰减在很大程度上是独立的噪声特性，但只取决于整体噪声功率。导出了闲波衰减率随噪声功率变化的解析表达式。对于模型验证，我们使用三个不同的超级计算平台上的微基准和模板算法。
英文摘要：
Most distributed-memory bulk-synchronous parallel programs in HPC assume that compute resources are available continuously and homogeneously across the allocated set of compute nodes. However, long one-off delays on individual processes can cause global disturbances, so-called idle waves, by rippling through the system. This process is mainly governed by the communication topology of the underlying parallel code. This paper makes significant contributions to the understanding of idle wave dynamics. We study the propagation mechanisms of idle waves across the ranks of MPI-parallel programs. We present a validated analytic model for their propagation velocity with respect to communication parameters and topology, with a special emphasis on sparse communication patterns. We study the interaction of idle waves with MPI collectives and show that, depending on the implementation, a collective may be transparent to the wave. Finally we analyze two mechanisms of idle wave decay: topological decay, which is rooted in differences in communication characteristics among parts of the system, and noise-induced decay, which is caused by system or application noise. We show that noise-induced decay is largely independent of noise characteristics but depends only on the overall noise power. An analytic expression for idle wave decay rate with respect to noise power is derived. For model validation we use microbenchmarks and stencil algorithms on three different supercomputing platforms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
