将嵌套并行模型扩展到具有可证明有效的并行机的嵌套数据流模型
Extending the Nested Parallel Model to the Nested Dataflow Model with Provably Efficient Schedulers
摘要：
嵌套并行（也称为fork-join）模型被广泛用于编写并行程序。然而，这两个组合构造，即“$\parallel$“（并行）和“$;$“（串行），不足以表达程序中的“部分依赖性”或“部分并行性”。本文提出了一种新的嵌套组合结构“$\leadsto$“，以处理器和高速缓存无关的方式表示算法中的部分依赖关系，从而将嵌套并行（NP）模型扩展到\emph{Nested Dataflow}（ND）模型.我们重新设计了几个分而治之的算法，从密集的线性代数的ND模型中的动态规划，并证明他们都有最佳的跨度，同时保持最佳的缓存复杂度。我们提出了运行时缓存器的设计，ND程序映射到多核处理器与多个级别的可能共享的高速缓存（即，并行存储器层次结构），并提供理论保证他们的能力，以保持局部性和负载平衡。为此，我们针对ND模型调整了空间有界（SB）调度器。我们表明，我们的算法增加了“并行化”的ND模型，SB的并行化可以使用额外的缓存未命中和运行时间上的渐近最优的界限比在NP模型的处理器数量更大。本文算法的运行时间为$O\left(\frac{\sum_{i=0}^{h-1} Q^{*}({\mathsf t};σ\cdot M_i)\cdot C_i}{p}\right)$，其中$Q^{*}$是任务${\mathsf t}$的缓存复杂度，$C_i$是$i$级缓存的未命中代价，$i$级缓存的大小为$M_i$，$σ\in(0,1)$是常数，$p$是$h$级缓存层次结构中的处理器数目.
英文摘要：
The nested parallel (a.k.a. fork-join) model is widely used for writing parallel programs. However, the two composition constructs, i.e. "$\parallel$" (parallel) and "$;$" (serial), are insufficient in expressing "partial dependencies" or "partial parallelism" in a program. We propose a new dataflow composition construct "$\leadsto$" to express partial dependencies in algorithms in a processor- and cache-oblivious way, thus extending the Nested Parallel (NP) model to the \emph{Nested Dataflow} (ND) model. We redesign several divide-and-conquer algorithms ranging from dense linear algebra to dynamic-programming in the ND model and prove that they all have optimal span while retaining optimal cache complexity. We propose the design of runtime schedulers that map ND programs to multicore processors with multiple levels of possibly shared caches (i.e, Parallel Memory Hierarchies) and provide theoretical guarantees on their ability to preserve locality and load balance. For this, we adapt space-bounded (SB) schedulers for the ND model. We show that our algorithms have increased "parallelizability" in the ND model, and that SB schedulers can use the extra parallelizability to achieve asymptotically optimal bounds on cache misses and running time on a greater number of processors than in the NP model. The running time for the algorithms in this paper is $O\left(\frac{\sum_{i=0}^{h-1} Q^{*}({\mathsf t};σ\cdot M_i)\cdot C_i}{p}\right)$, where $Q^{*}$ is the cache complexity of task ${\mathsf t}$, $C_i$ is the cost of cache miss at level-$i$ cache which is of size $M_i$, $σ\in(0,1)$ is a constant, and $p$ is the number of processors in an $h$-level cache hierarchy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
