arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 0909.4341cs.DS
收藏

外部存储器中的轻量级数据索引和压缩

Lightweight Data Indexing and Compression in External Memory

Paolo Ferragina, Travis Gagie, Giovanni Manzini

摘要:

在本文中,我们描述的算法计算的BWT和建设(压缩)索引在外部存储器中。我们的算法的创新特点是,他们是轻量级的意义上说,对于一个输入的大小$n$,他们只使用${n}$位的磁盘工作空间,而所有以前的方法使用$\Th{n \log n}$位的磁盘工作空间。此外,我们的算法只通过顺序扫描访问磁盘数据,因此它们充分利用了现代磁盘功能,使顺序磁盘访问比随机访问快得多。我们还提出了一个基于扫描的算法,用于反相的BWT,使用$\Th{n}$位的工作空间,和一个轻量级的{ \em内部存储器}算法计算的BWT,这是最快的文献中可用的工作空间时,$\os{n}$位。最后,我们证明了{ \em下限}的计算和反转的BWT通过顺序扫描的经典产品的复杂性:内部存储器空间$\times$通过磁盘数据的数量。

英文摘要:

In this paper we describe algorithms for computing the BWT and for building (compressed) indexes in external memory. The innovative feature of our algorithms is that they are lightweight in the sense that, for an input of size $n$, they use only ${n}$ bits of disk working space while all previous approaches use $\Th{n \log n}$ bits of disk working space. Moreover, our algorithms access disk data only via sequential scans, thus they take full advantage of modern disk features that make sequential disk accesses much faster than random accesses. We also present a scan-based algorithm for inverting the BWT that uses $\Th{n}$ bits of working space, and a lightweight {\em internal-memory} algorithm for computing the BWT which is the fastest in the literature when the available working space is $\os{n}$ bits. Finally, we prove {\em lower} bounds on the complexity of computing and inverting the BWT via sequential scans in terms of the classic product: internal-memory space $\times$ number of passes over the disk data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑