ROOT树I/O的演化
Evolution of the ROOT Tree I/O
摘要:
ROOT TTree数据格式编码数百PB的高能和核物理事件。它的列式布局驱动了快速分析,因为只有那些在给定分析中真正使用的部分(“分支”)才需要从存储中读取。它的独特功能是无缝的C++集成,允许用户直接存储他们的事件类,而无需显式定义数据模式。在这篇文章中,我们介绍了未来ROOT 7事件I/O的现状和计划。随着ROOT 7接口的现代化,我们的目标是健壮的,在可能的编译时安全的C++接口读取和写入事件数据。在性能方面,我们展示了使用ROOT新的实验I/O子系统的第一个基准测试,该子系统将最好的TTrees与列式数据格式的最新进展相结合。一个核心要素是高级逻辑数据布局(C++类)与低级物理数据布局(简单类型的存储支持嵌套向量)的强大分离。我们展示了新的,优化的物理数据布局如何加快串行化和并行化,并促进并行,矢量化和批量操作。这使得ROOT I/O能够在即将推出的超快NVRAM存储设备以及对象存储等无文件存储系统上以最佳方式运行。
英文摘要:
The ROOT TTree data format encodes hundreds of petabytes of High Energy and Nuclear Physics events. Its columnar layout drives rapid analyses, as only those parts ("branches") that are really used in a given analysis need to be read from storage. Its unique feature is the seamless C++ integration, which allows users to directly store their event classes without explicitly defining data schemas. In this contribution, we present the status and plans of the future ROOT 7 event I/O. Along with the ROOT 7 interface modernization, we aim for robust, where possible compile-time safe C++ interfaces to read and write event data. On the performance side, we show first benchmarks using ROOT's new experimental I/O subsystem that combines the best of TTrees with recent advances in columnar data formats. A core ingredient is a strong separation of the high-level logical data layout (C++ classes) from the low-level physical data layout (storage backed nested vectors of simple types). We show how the new, optimized physical data layout speeds up serialization and deserialization and facilitates parallel, vectorized and bulk operations. This lets ROOT I/O run optimally on the upcoming ultra-fast NVRAM storage devices, as well as file-less storage systems such as object stores.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。