arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv cs/0306092cs.DC
收藏

利用网格数据场和ROOT I/O构建高性能并行文件系统

Building A High Performance Parallel File System Using Grid Datafarm and ROOT I/O

Y. Morita, H. Sato, Y. Watase, O. Tatebe, S. Sekiguchi, S. Matsuoka, N. Soda, A. Dell'Acqua

摘要:

在大型强子对撞机实验中预见到的PB级数据量要求在全球范围的分布式计算中仔细考虑持久化设计和系统设计。HENP数据分析的事件并行性使我们能够在数据处理阶段、数据管理阶段和数据传输阶段保持并行性的同时,最大限度地利用高性能集群计算和网络。FADS/ Goofy是Geant 4的多功能检测器模拟框架,其模块化架构使您可以轻松选择Objectivity/DB和ROOT I/O等持久性技术的插件设施。该框架的设计自然与网格数据农场(Gfarm)的并行文件系统。FADS/Goofy被证明可以使用512 CPU的PC集群生成10^6个Geant 4模拟的Atlas Mockup事件。使用Gfarm文件系统复制ROOT I/O文件中的数据。直方图信息是从分布式ROOT文件中收集的。在数据复制过程中,在美国和日本的七个参与PC集群中,PC集群之间的数据传送率已被证明超过2.3 Gbps。

英文摘要:

Sheer amount of petabyte scale data foreseen in the LHC experiments require a careful consideration of the persistency design and the system design in the world-wide distributed computing. Event parallelism of the HENP data analysis enables us to take maximum advantage of the high performance cluster computing and networking when we keep the parallelism both in the data processing phase, in the data management phase, and in the data transfer phase. A modular architecture of FADS/ Goofy, a versatile detector simulation framework for Geant4, enables an easy choice of plug-in facilities for persistency technologies such as Objectivity/DB and ROOT I/O. The framework is designed to work naturally with the parallel file system of Grid Datafarm (Gfarm). FADS/Goofy is proven to generate 10^6 Geant4-simulated Atlas Mockup events using a 512 CPU PC cluster. The data in ROOT I/O files is replicated using Gfarm file system. The histogram information is collected from the distributed ROOT files. During the data replication it has been demonstrated to achieve more than 2.3 Gbps data transfer rate between the PC clusters over seven participating PC clusters in the United States and in Japan.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑