arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.07599cs.DScs.IRcs.ITmath.IT
收藏

在重复文本集合上快速、小巧、简单地列出文档

Fast, Small, and Simple Document Listing on Repetitive Text Collections

Dustin Cobas, Gonzalo Navarro

摘要:

在字符串集合上列出文档的任务是查找出现模式的所有文档。它被认为是最基本的文档检索问题,并在各种应用中是有用的。许多增长最快的字符串集合由非常相似的文档组成,例如版本化代码和文档集合、基因组存储库等。为重复文本集合设计的普通模式匹配索引实现了空间数量级的减少。相反,用于文档检索的类似索引并不多。在本文中,我们提出了一个简单的文件列表索引的重复字符串集合的总长度$n$,列出$ndoc$不同的文件中出现的模式的长度$m$在时间$\mathcal{O}(m+ndoc \cdot \log n)$。我们利用文档数组的重复性(即,粗化为文档标识符的后缀数组)来对其进行语法压缩,同时预计算对非终结符的回答,并且也以语法压缩的形式存储它们。我们的实验结果表明,我们的索引大大优于现有的替代品在空间/时间权衡地图。

英文摘要:

Document listing on string collections is the task of finding all documents where a pattern appears. It is regarded as the most fundamental document retrieval problem, and is useful in various applications. Many of the fastest-growing string collections are composed of very similar documents, such as versioned code and document collections, genome repositories, etc. Plain pattern-matching indexes designed for repetitive text collections achieve orders-of-magnitude reductions in space. Instead, there are not many analogous indexes for document retrieval. In this paper we present a simple document listing index for repetitive string collections of total length $n$ that lists the $ndoc$ distinct documents where a pattern of length $m$ appears in time $\mathcal{O}(m+ndoc \cdot \log n)$. We exploit the repetitiveness of the document array (i.e., the suffix array coarsened to document identifiers) to grammar-compress it while precomputing the answers to nonterminals, and store them in grammar-compressed form as well. Our experimental results show that our index sharply outperforms existing alternatives in the space/time tradeoff map.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑