arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.09043physics.data-ancs.DBphysics.acc-phphysics.comp-ph
收藏

RNTuple性能:状态和展望

RNTuple performance: Status and Outlook

Javier Lopez-Gomez, Jakob Blomer

摘要:

即将进行的HEP实验,例如在HL-LHC,预计将增加至少一个数量级的生成数据量。为了保持分析大量数据的能力,充分利用现代存储硬件和系统(如低延迟高带宽NVMe设备和分布式对象存储)变得至关重要。为此,ROOT RNTuple I/O子系统被设计为解决ROOT当前最先进的TTree I/O子系统的性能瓶颈和缺点。RNTuple提供了对TTree二进制格式和访问API的向后不兼容的重新设计,该API改进了ROOT事件数据I/O,以应对未来几十年的挑战。它专注于紧凑的数据格式,现代存储硬件的性能工程,例如通过默认进行并行和异步I/O调用,以及易于正确使用的强大接口。在这篇文章中,我们评估了典型的HEP分析任务的RNTuple性能。我们将RNTuple提供的吞吐量与HEP之外的流行I/O库(如HDF 5和Apache Parquet)进行了比较。我们展示了RNTuple在HEP分析工作流程中的优势,并展望了其在生产中的应用。

英文摘要:

Upcoming HEP experiments, e.g. at the HL-LHC, are expected to increase the volume of generated data by at least one order of magnitude. In order to retain the ability to analyze the influx of data, full exploitation of modern storage hardware and systems, such as low-latency high-bandwidth NVMe devices and distributed object stores, becomes critical. To this end, the ROOT RNTuple I/O subsystem has been designed to address performance bottlenecks and shortcomings of ROOT's current state of the art TTree I/O subsystem. RNTuple provides a backwards-incompatible redesign of the TTree binary format and access API that evolves the ROOT event data I/O for the challenges of the upcoming decades. It focuses on a compact data format, on performance engineering for modern storage hardware, for instance through making parallel and asynchronous I/O calls by default, and on robust interfaces that are easy to use correctly. In this contribution, we evaluate the RNTuple performance for typical HEP analysis tasks. We compare the throughput delivered by RNTuple to popular I/O libraries outside HEP, such as HDF5 and Apache Parquet. We demonstrate the advantages of RNTuple for HEP analysis workflows and provide an outlook on the road to its use in production.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑