arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.01032cs.IRcs.DS
收藏

关于切片排序的序列

On Slicing Sorted Integer Sequences

Giulio Ermanno Pibiri

摘要:

在小空间中表示有序整数序列是Web搜索引擎等大规模检索系统的核心问题。高效的查询解决方案,例如,交叉或随机接入是通过仔细划分序列来实现的。在这项工作中,我们描述和比较两种不同的分区范式:分区基数和分区宇宙。虽然这些范例背后的思想多年来在编码和算法社区中已经是已知的,但是倒排索引压缩已经广泛地采用了前一种范例,而后者只受到很少的关注。结果,对于倒排索引压缩的设置,缺少这两者之间的实验比较。我们还提出并实现了一个解决方案,递归切片的宇宙表示的序列,以实现紧凑的存储,并达到快速查询执行。虽然比一些最先进的表示更大,这种切片方法大大提高了列表交集和并集的性能,同时在压缩空间中操作,从而为问题提供了一个很好的空间/时间权衡。

英文摘要:

Representing sorted integer sequences in small space is a central problem for large-scale retrieval systems such as Web search engines. Efficient query resolution, e.g., intersection or random access, is achieved by carefully partitioning the sequences. In this work we describe and compare two different partitioning paradigms: partitioning by cardinality and partitioning by universe. Although the ideas behind such paradigms have been known in the coding and algorithmic community since many years, inverted index compression has extensively adopted the former paradigm, whereas the latter has received only little attention. As a result, an experimental comparison between these two is missing for the setting of inverted index compression. We also propose and implement a solution that recursively slices the universe of representation of a sequence to achieve compact storage and attain to fast query execution. Albeit larger than some state-of-the-art representations, this slicing approach substantially improves the performance of list intersections and unions while operating in compressed space, thus offering an excellent space/time trade-off for the problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑