Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to many events, but only a subset of the information stored for each of them. Future colliders, and particularly HL-LHC, will bring an increase of at least one order of magnitude in the volume of generated data. Therefore, the use of modern storage hardware, such as low-latency high-bandwidth NVMe devices and distributed object stores, becomes more important. However, TTree was not designed to optimally exploit modern hardware and may become a bottleneck for data retrieval. The ROOT RNTuple I/O system aims at overcoming TTree's limitations and at providing improved efficiency for modern storage systems. In this paper, we extend RNTuple with a backend that uses Intel DAOS as the underlying storage, demonstrating that the RNTuple architecture can accommodate high-performance object stores. From the user perspective, data can be accessed with minimal changes to the code, that is by replacing a filesystem path by a DAOS URI. Our performance evaluation shows that the new backend can be used for realistic analyses, while outperforming the compatibility solution provided by the DAOS project.
CobbleDB: Modelling Levelled Storage by Composition
University of British Columbia(不列颠哥伦比亚大学);Télécom SudParis(巴黎电信学院);RPTU University of Kaiserslautern-Landau(凯泽斯劳滕-兰道理工大学);Sorbonne-Université—LIP6 & Inria(索邦大学—LIP6与法国国家信息与自动化研究所)
A High-Performance Persistent Memory Key-Value Store with Near-Memory Compute
MCAS(Memory Centric Active Storage)是一种用于高性能持久数据存储的持久内存层。它从一开始就设计为通过内存持久性和复制提供具有低延迟保证和数据持久性的键值功能。为了减少数据移动并进一步提高性能,我们提供了对用户定义的“下推”操作(称为活动数据对象)的支持,这些操作可以在与一个或多个键相关联的值内存上直接安全地执行。ADO机制允许复杂的基于指针的动态数据结构(例如,树)被存储在持久存储器中并对其进行操作。为此,我们研究了MCAS-ADO在处理企业存储系统元数据以实现连续数据保护(CDP)方面的实际用例。这需要不断更新复杂的元数据,必须保持一致和持久。在本文中,我们i.)提出了MCAS-ADO系统的体系结构;展示CDP用例是如何实现的,最后iii.)在此用例的上下文中对系统性能进行评估。