arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 1704.06976cs.DC
收藏

探索ROOT IO的压缩技术

Exploring compression techniques for ROOT IO

Zhe Zhang, Brian Bockelman

摘要:

ROOT提供了一种在整个HEP社区中使用的灵活格式。用例的数量-从归档数据格式到最终阶段分析-需要向用户展示一些权衡。例如,传统DEFLATE算法中的高“压缩级别”将导致较小的文件(节省磁盘空间),代价是较慢的解压缩(读取时消耗CPU时间)。在大型强子对撞机实验的规模上,糟糕的设计选择可能会导致浪费TB级的空间或浪费CPU时间。我们探索并试图量化其中的一些权衡。具体而言,我们探讨:使用替代压缩算法来优化读取性能;压缩单个事件的替代方法以允许有效的随机访问;以及整个文件压缩的新方法。给出了定量结果,以及如何针对不同用例做出压缩决策的指导。

英文摘要:

ROOT provides an flexible format used throughout the HEP community. The number of use cases - from an archival data format to end-stage analysis - has required a number of tradeoffs to be exposed to the user. For example, a high "compression level" in the traditional DEFLATE algorithm will result in a smaller file (saving disk space) at the cost of slower decompression (costing CPU time when read). At the scale of the LHC experiment, poor design choices can result in terabytes of wasted space or wasted CPU time. We explore and attempt to quantify some of these tradeoffs. Specifically, we explore: the use of alternate compressing algorithms to optimize for read performance; an alternate method of compressing individual events to allow efficient random access; and a new approach to whole-file compression. Quantitative results are given, as well as guidance on how to make compression decisions for different use cases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑