arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1608.04431cs.DCcs.DS
收藏

离散星系团上三胞数字高程模式的并行无辐散流积累

Parallel Non-divergent Flow Accumulation For Trillion Cell Digital Elevation Models On Desktops Or Clusters

Richard Barnes

摘要:

大陆尺度的数据集挑战水文算法处理数字高程模型。流量累积是许多此类算法的重要输入;在这里,我将其计算并行化。新算法可以在一个或多个核心或多台机器上工作,并且可以利用大内存或处理小内存。与以前的并行算法不同,新算法保证了每个栅格单元的存储器访问和通信事件的固定数量。在测试中,新算法运行速度更快,使用的资源比以前的算法更少,表现出约30%的强和弱扩展效率高达48个核心,并在数据集上线性扩展超过三个数量级。测试的最大数据集有两万亿(2*10^12)个细胞。对于48个核心,处理需要24分钟的壁时间(14.5计算小时)。该测试比文献中先前进行的任何测试大三个数量级。完整的、注释良好的源代码和正确性测试可以在Github上找到。

英文摘要:

Continent-scale datasets challenge hydrological algorithms for processing digital elevation models. Flow accumulation is an important input for many such algorithms; here, I parallelize its calculation. The new algorithm works on one or many cores, or multiple machines, and can take advantage of large memories or cope with small ones. Unlike previous parallel algorithms, the new algorithm guarantees a fixed number of memory access and communication events per raster cell. In testing, the new algorithm ran faster and used fewer resources than previous algorithms, exhibiting ~30% strong and weak scaling efficiencies up to 48 cores and linear scaling across datasets ranging over three orders of magnitude. The largest dataset tested had two trillion (2*10^12) cells. With 48 cores, processing required 24 minutes wall-time (14.5 compute-hours). This test is three orders of magnitude larger than any previously performed in the literature. Complete, well-commented source code and correctness tests are available on Github.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑