arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.07304cs.DBhep-ex
收藏

探索高能物理数据存储的对象存储

Exploring Object Stores for High-Energy Physics Data Storage

Javier López-Gómez, Jakob Blomer

摘要:

在过去的二十年里,ROOT TTree已被用于存储超过1 EB的高能物理(HEP)事件。已经证明,TTree列式磁盘布局对于通常需要访问许多事件但仅访问为每个事件存储的信息的子集的HEP数据的分析是理想的。未来的对撞机,特别是HL-LHC,将使生成的数据量增加至少一个数量级。因此,使用现代存储硬件,如低延迟高带宽NVMe设备和分布式对象存储变得更加重要。然而,TTree并没有被设计成最佳地利用现代硬件,并且可能成为数据检索的瓶颈。ROOT RNTuple I/O系统旨在克服TTree的局限性,并为现代存储系统提供更高的效率。在本文中,我们通过使用英特尔DAOS作为底层存储的后端来扩展RNTuple,从而证明RNTuple架构可以容纳高性能的对象存储。从用户的角度来看,只需对代码进行最小的更改就可以访问数据,也就是说,通过用DAOS URI替换文件系统路径。我们的性能评估表明,新的后端可以用于现实的分析,同时优于DAOS项目提供的兼容性解决方案。

英文摘要:

Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to many events, but only a subset of the information stored for each of them. Future colliders, and particularly HL-LHC, will bring an increase of at least one order of magnitude in the volume of generated data. Therefore, the use of modern storage hardware, such as low-latency high-bandwidth NVMe devices and distributed object stores, becomes more important. However, TTree was not designed to optimally exploit modern hardware and may become a bottleneck for data retrieval. The ROOT RNTuple I/O system aims at overcoming TTree's limitations and at providing improved efficiency for modern storage systems. In this paper, we extend RNTuple with a backend that uses Intel DAOS as the underlying storage, demonstrating that the RNTuple architecture can accommodate high-performance object stores. From the user perspective, data can be accessed with minimal changes to the code, that is by replacing a filesystem path by a DAOS URI. Our performance evaluation shows that the new backend can be used for realistic analyses, while outperforming the compatibility solution provided by the DAOS project.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑