arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.18004cs.DC
收藏

对数时间内最优广播调度及其在广播、全广播、归约和全归约中的应用

Optimal Broadcast Schedules in Logarithmic Time with Applications to Broadcast, All-Broadcast, Reduction and All-Reduction

Jesper Larsson Träff

AI总结:

本文提出最优 O(log p) 时间算法计算并行计算系统中的轮次最优广播调度,并扩展至归约、全广播和全归约,显著改进了先前复杂度并应用于 MPI 集合通信操作。

摘要:

我们给出了计算消息传递并行计算系统中轮次最优广播调度的最优快速 O(log p) 时间(每处理器)算法。这肯定地回答了 SPAA 2022 BA 和一篇 CLUSTER 2022 论文中提出的难题。我们观察到,计算出的调度和循环通信图同样可用于归约、全广播和全归约,从而为这些问题带来了新的轮次最优算法。这些观察肯定地回答了一篇 CLUSTER 2023 论文中提出的开放问题。该问题是在具有全双向、单端口通信能力的 p 处理器全连接网络(或其子图)中,将 n 个不可分割的数据块从给定根处理器广播到所有其他处理器。在此模型中,需要 n-1+⌈log₂ p⌉ 通信轮次。我们的新算法为网络中的每个处理器计算接收和发送调度,每个大小为 ⌈log₂ p⌉,在 O(1) 时间内唯一确定每轮中处理器将接收的新数据块及其必须发送的已接收数据块。调度计算由每个处理器独立完成,无需通信。广播通信子图是一个易于计算的、有向的 ⌈log₂ p⌉ 正则循环图,该图在其他地方亦有使用。我们展示了如何在 O(log p) 的最优时间和空间内完成调度计算,较先前 O(p log² p) 和 O(log³ p) 的结果有显著改进。调度计算和广播算法易于实现,但其正确性和复杂性并不明显。这些调度用于 MPI(Message-Passing Interface)集合通信操作 MPI_Bcast、MPI_Allgatherv、MPI_Reduce 和 MPI_Reduce_scatter 的新实现。文中给出了初步的实验结果。

英文摘要:

We give optimally fast $O(\log p)$ time (per processor) algorithms for computing round-optimal broadcast schedules for message-passing parallel computing systems. This affirmatively answers difficult questions posed in a SPAA 2022 BA and a CLUSTER 2022 paper. We observe that the computed schedules and circulant communication graph can likewise be used for reduction, all-broadcast and all-reduction as well, leading to new, round-optimal algorithms for these problems. These observations affirmatively answer open questions posed in a CLUSTER 2023 paper. The problem is to broadcast $n$ indivisible blocks of data from a given root processor to all other processors in a (subgraph of a) fully connected network of $p$ processors with fully bidirectional, one-ported communication capabilities. In this model, $n-1+\lceil\log_2 p\rceil$ communication rounds are required. Our new algorithms compute for each processor in the network receive and send schedules each of size $\lceil\log_2 p\rceil$ that determine uniquely in $O(1)$ time for each communication round the new block that the processor will receive, and the already received block it has to send. Schedule computations are done independently per processor without communication. The broadcast communication subgraph is an easily computable, directed, $\lceil\log_2 p\rceil$-regular circulant graph also used elsewhere. We show how the schedule computations can be done in optimal time and space of $O(\log p)$, improving significantly over previous results of $O(p\log^2 p)$ and $O(\log^3 p)$, respectively. The schedule computation and broadcast algorithms are simple to implement, but correctness and complexity are not obvious. The schedules are used for new implementations of the MPI (Message-Passing Interface) collectives MPI_Bcast, MPI_Allgatherv, MPI_Reduce and MPI_Reduce_scatter. Preliminary experimental results are given.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑