arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.01380cs.CE
收藏

大规模计算流体力学数据的任务并行实时压缩

Task-parallel in-situ temporal compression of large-scale computational fluid dynamics data

Heather Pacella, Alec Dunton, Alireza Doostan, Gianluca Iaccarino

摘要:

当今的计算流体动力学模拟生成非常大量的数据,有时在TB/s的数量级。通常,这些数据的很大一部分被丢弃,因为当前的存储系统无法跟上。为了解决这个问题,可以将数据压缩算法应用于包含感兴趣的流量的数据阵列,以减少总存储量。压缩方法或者精确地重建原始数据集(无损压缩),或者提供原始数据集的近似表示(有损压缩)。矩阵列插值分解(ID)可以被实现为用于数据矩阵的一种类型的有损压缩,其将原始数据矩阵因子化为两个较小因子矩阵的乘积。这些矩阵中的一个由原始数据矩阵的列的子集组成,而另一个是系数矩阵,其将原始数据矩阵的列近似为所选列的线性组合。激励这项工作的观察,ID算法的结构,使他们自然适合异步性质的基于任务的并行性,他们能够独立运作的子域的系统的利益,因此,提供不同程度的压缩。采用基于任务的Legion编程模型,实现了一种适用于CFD应用的单通道ID算法(SPID)。性能研究,可扩展性,和压缩算法的准确性的分析泰勒-格林涡的问题,其次是大规模实施的可压缩泰勒-格林涡使用高阶Navier-Stokes求解器。在这两种情况下,超过100的压缩因子都是在相对误差等于或低于10 e-3时实现的。此外,强和弱的缩放结果表明,引入SPID求解器导致运行时间的增加可以忽略不计。

英文摘要:

Present day computational fluid dynamics simulations generate extremely large amounts of data, sometimes on the order of TB/s. Often, a significant fraction of this data is discarded because current storage systems are unable to keep pace. To address this, data compression algorithms can be applied to data arrays containing flow quantities of interest to reduce the overall amount of storage. Compression methods either exactly reconstruct the original dataset (lossless compression) or provide an approximate representation of the original dataset (lossy compression). The matrix column interpolative decomposition (ID) can be implemented as a type of lossy compression for data matrices that factors the original data matrix into a product of two smaller factor matrices. One of these matrices consists of a subset of the columns of the original data matrix, while the other is a coefficient matrix which approximates the columns of the original data matrix as linear combinations of the selected columns. Motivating this work is the observation that the structure of ID algorithms makes them a natural fit for the asynchronous nature of task-based parallelism; they are able to operate independently on sub-domains of the system of interest and, as a result, provide varied levels of compression. Using the task-based Legion programming model, a single-pass ID algorithm (SPID) for CFD applications is implemented. Performance studies, scalability, and the accuracy of the compression algorithms are presented for an analytical Taylor-Green vortex problem, followed by a large-scale implementation of a compressible Taylor-Green vortex using a high-order Navier-Stokes solver. In both cases, compression factors exceeding 100 are achieved with relative errors at or below 10e-3. Moreover, strong and weak scaling results demonstrate that introducing SPID to solvers leads to negligible increases in runtime.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑