arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2507.04785cs.DC
收藏

通信轮次与计算高效的排他前缀和算法(用于 MPI_Exscan)

Communication Round and Computation Efficient Exclusive Prefix-Sums Algorithms (for MPI_Exscan)

Jesper Larsson Träff

AI总结:

本文针对 MPI_Exscan 排他前缀和问题,提出一种新的简单算法,在 q=⌈log₂(p-1)+log₂(4/3)⌉ 轮通信内仅用 q-1 次 ⊕ 运算完成计算,并在 36 节点集群上验证其相对现有实现具有改进潜力。

摘要:

并行扫描原语在结合二元运算符 $\oplus$ 下,对由 $p$ 个连续排名的处理器贡献的输入向量逐元素计算包含或排他前缀和。在具有有界单端口通信能力的消息传递系统中,执行扫描至少需要 $\lceil\log_2 p\rceil$ 或 $\lceil\log_2 (p-1)\rceil$ 轮通信。虽然对于包含扫描已有众所周知且简单的算法,可在 $\lceil\log_2 p\rceil$ 轮通信内解决问题,并应用 $\lceil\log_2 p\rceil$ 次 $\oplus$(这可能代价高昂),但排他扫描似乎更为困难。传统上,该问题要么通过 $\lceil\log_2 (p-1)\rceil+1$ 轮通信解决(例如通过移位输入向量),要么在 $\lceil\log_2 p\rceil$ 轮通信内使用 $2\lceil\log_2 p\rceil-1$ 次 $\oplus$ 应用(通过修改的包含扫描算法)解决。我们给出一种新的简单算法,在 $q=\lceil\log_2 (p-1)+\log_2\frac{4}{3}\rceil$ 轮同时发送-接收通信中计算排他前缀和,并应用 $q-1$ 次 $\oplus$。我们在一个配备最先进 MPI 库的 36 节点小型集群上,将三种用 MPI 实现的算法与 MPI 库原生的 MPI\\_Exscan 原语进行比较,结果表明对标准实现进行改进是可能且有价值的。这些算法假设输入向量较小,因此性能主要由通信轮数决定。对于大型输入向量,必须使用其他(流水线、固定度树)算法。

英文摘要:

Parallel scan primitives compute element-wise inclusive or exclusive prefix sums of input vectors contributed by $p$ consecutively ranked processors under an associative, binary operator $\oplus$. In message-passing systems with bounded, one-ported communication capabilities, at least $\lceil\log_2 p\rceil$ or $\lceil\log_2 (p-1)\rceil$ communication rounds are required to perform the scans. While there are well-known, simple algorithms for the inclusive scan that solve the problem in $\lceil\log_2 p\rceil$ communication rounds with $\lceil\log_2 p\rceil$ applications of $\oplus$ (which could be expensive), the exclusive scan appears more difficult. Conventionally, the problem is solved with either $\lceil\log_2 (p-1)\rceil+1$ communication rounds (e.g., by shifting the input vectors), or in $\lceil\log_2 p\rceil$ communication rounds with $2\lceil\log_2 p\rceil-1$ applications of $\oplus$ (by a modified inclusive scan algorithm). We give a new, simple algorithm that computes the exclusive prefix sums in $q=\lceil\log_2 (p-1)+\log_2\frac{4}{3}\rceil$ simultaneous send-receive communication rounds with $q-1$ applications of $\oplus$. We compare the three algorithms implemented in MPI against the MPI library native MPI\_Exscan primitive on a small, $36$-node cluster with a state-of-the-art MPI library, indicating possible and worthwhile improvements to standard implementations. The algorithms assume input vectors to be small so that performance is dominated by the number of communication rounds. For large input vectors, other (pipelined, fixed-degree tree) algorithms must be used.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑