arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.05975cs.DC
收藏

线性代数核的并行I/O最优性:近优LU分解

On the Parallel I/O Optimality of Linear Algebra Kernels: Near-Optimal LU Factorization

Grzegorz Kwasniewski, Tal Ben-Nun, Alexandros Nikolaos Ziogas, Timo Schneider, Maciej Besta, Torsten Hoefler

摘要:

稠密线性代数内核,如线性求解器或张量收缩,是许多科学计算应用的基本组成部分。在这项工作中,我们提出了一种新的方法,推导出并行I/O的下限,这个广泛的家庭的程序。基于X分区抽象,我们的方法显式捕获语句间的依赖关系。应用我们的分析LU分解,我们得出COnfLUX,LU算法的并行I/O成本的$N^3 / (P \sqrt{M})$通信元素每个处理器-只有$1/3\times$在我们建立的下限。我们评估COnfLUX对各种问题的大小,展示经验的结果,符合我们的理论分析,通信渐近小于Cray ScaLAPACK或SLATE,并优于渐近最优的CANDMC库。在Piz Daint的$1$、$024$节点上运行,COnfLUX的$\times$通信量比第二个最佳实现少1.6 $\times$,预计在Summit上全面运行时将少2.1 $\times$。

英文摘要:

Dense linear algebra kernels, such as linear solvers or tensor contractions, are fundamental components of many scientific computing applications. In this work, we present a novel method of deriving parallel I/O lower bounds for this broad family of programs. Based on the X-partitioning abstraction, our method explicitly captures inter-statement dependencies. Applying our analysis to LU factorization, we derive COnfLUX, an LU algorithm with the parallel I/O cost of $N^3 / (P \sqrt{M})$ communicated elements per processor -- only $1/3\times$ over our established lower bound. We evaluate COnfLUX on various problem sizes, demonstrating empirical results that match our theoretical analysis, communicating asymptotically less than Cray ScaLAPACK or SLATE, and outperforming the asymptotically-optimal CANDMC library. Running on $1$,$024$ nodes of Piz Daint, COnfLUX communicates 1.6$\times$ less than the second-best implementation and is expected to communicate 2.1$\times$ less on a full-scale run on Summit.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑