arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.10072cs.DC
收藏

MPI中广播与不规则Allgather的轮次最优$n$-块广播调度的(多)对数时间构造

(Poly)Logarithmic Time Construction of Round-optimal $n$-Block Broadcast Schedules for Broadcast and irregular Allgather in MPI

Jesper Larsson Träff

AI总结:

提出一种快速、无通信、并行的轮次最优$n$-块广播调度构造方法,改进了现有顺序构造的时空复杂度,并用于实现MPI中广播与不规则allgather的通信最优算法,显著提升了标准MPI库的性能。

摘要:

我们提出了一种快速、无通信、并行的最优通信调度构造方法,该调度允许在具有全双向通信的1端口、$p$处理器网络中,将$n$个不同的数据块从根处理器广播到所有其他处理器。对于任意的$p$和$n$,在该模型中广播需要$n-1+\lceil\log_2 p\rceil$个通信轮次。与其他构造方法不同,所有处理器遵循相同的循环图通信模式,这使得该调度也可用于allgather(全对全广播)操作。这种新构造每个处理器耗时$O(\log^3 p)$步,每个处理器可以在$O(\log p)$空间中独立于其他处理器计算其调度部分。该结果显著改进了Träff和Ripke(2009)的顺序$O(p \log^2 p)$时间和$O(p\log p)$空间构造,具有重要的实用价值。该轮次最优调度构造随后被用于实现MPI(Message-Passing Interface)中广播和(不规则)allgather集合操作的通信最优算法,并在某些问题范围内显著且实用地改进了标准MPI库(\texttt{mpich},OpenMPI,Intel MPI)中的实现。对不规则allgather操作的应用是全新的。

英文摘要:

We give a fast(er), communication-free, parallel construction of optimal communication schedules that allow broadcasting of $n$ distinct blocks of data from a root processor to all other processors in $1$-ported, $p$-processor networks with fully bidirectional communication. For any $p$ and $n$, broadcasting in this model requires $n-1+\lceil\log_2 p\rceil$ communication rounds. In contrast to other constructions, all processors follow the same, circulant graph communication pattern, which makes it possible to use the schedules for the allgather (all-to-all-broadcast) operation as well. The new construction takes $O(\log^3 p)$ time steps per processor, each of which can compute its part of the schedule independently of the other processors in $O(\log p)$ space. The result is a significant improvement over the sequential $O(p \log^2 p)$ time and $O(p\log p)$ space construction of Träff and Ripke (2009) with considerable practical import. The round-optimal schedule construction is then used to implement communication optimal algorithms for the broadcast and (irregular) allgather collective operations as found in MPI (the \emph{Message-Passing Interface}), and significantly and practically improves over the implementations in standard MPI libraries (\texttt{mpich}, OpenMPI, Intel MPI) for certain problem ranges. The application to the irregular allgather operation is entirely new.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑