SimFS:一个仿真数据虚拟化文件系统接口
SimFS: A Simulation Data Virtualizing File System Interface
摘要:
如今,模拟可以产生PB级的数据,存储在并行文件系统或大型数据库中。这些数据在几十年的时间里经常被成千上万的分析师和科学家访问。然而,长时间存储这些数据量并不具有成本效益,在某些情况下,实际上是不可能的。我们建议透明地虚拟化模拟数据,通过不存储完整的输出和按需重新模拟丢失的数据来放松存储要求。我们开发SimFS,一个文件系统接口,它将模拟输出的虚拟化视图暴露给分析应用程序并管理重新模拟。SimFS监控分析应用程序的访问模式,以便(1)决定为更快的访问而存储的数据,以及(2)采用预取策略来减少丢失数据的访问时间。虚拟化仿真数据允许我们用存储交换计算:这种模式变得类似于传统的磁盘分析(所有数据都存储)或原位分析(没有数据存储),这取决于分配给SimFS的存储资源。总的来说,通过利用不断增长的计算能力和放宽存储容量的要求,SimFS提供了一个可行的途径,以exa规模的模拟。
英文摘要:
Nowadays simulations can produce petabytes of data to be stored in parallel filesystems or large-scale databases. This data is accessed over the course of decades often by thousands of analysts and scientists. However, storing these volumes of data for long periods of time is not cost effective and, in some cases, practically impossible. We propose to transparently virtualize the simulation data, relaxing the storage requirements by not storing the full output and re-simulating the missing data on demand. We develop SimFS, a file system interface that exposes a virtualized view of the simulation output to the analysis applications and manages the re-simulations. SimFS monitors the access patterns of the analysis applications in order to (1) decide the data to keep stored for faster accesses and (2) to employ prefetching strategies to reduce the access time of missing data. Virtualizing simulation data allows us to trade storage for computation: this paradigm becomes similar to traditional on-disk analysis (all data is stored) or in situ (no data is stored) according with the storage resources that are assigned to SimFS. Overall, by exploiting the growing computing power and relaxing the storage capacity requirements, SimFS offers a viable path towards exa-scale simulations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。