arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2004.02781cs.DS
收藏

索引高度重复的字符串集合

Indexing Highly Repetitive String Collections

Gonzalo Navarro

摘要:

二十年前,索引字符串集合的突破使得在压缩空间内表示它们成为可能,同时提供索引搜索功能。随着这项新技术渗透到生物信息学等应用中,字符串集合的增长超过了摩尔定律,并挑战了我们处理它们的能力,即使是压缩形式。幸运的是,事实证明,这些快速增长的字符串集合中有许多是高度重复的,因此它们的信息量比它们的普通大小低几个数量级。然而,用于经典集合的统计压缩方法对这种重复性视而不见,因此开发了一套新的技术来适当地利用它。由此产生的索引形成了新一代的数据结构,能够处理我们所面临的巨大的重复字符串集合。在本综述中,我们将介绍导致这些数据结构的算法发展。我们描述了不同的压缩范式,已被用来利用重复性,基本的算法思想,形成所有现有的索引的基础,并已提出的各种结构,比较它们在理论和实践方面。最后,我们谈谈这一迷人领域目前面临的挑战。

英文摘要:

Two decades ago, a breakthrough in indexing string collections made it possible to represent them within their compressed space while at the same time offering indexed search functionalities. As this new technology permeated through applications like bioinformatics, the string collections experienced a growth that outperforms Moore's Law and challenges our ability of handling them even in compressed form. It turns out, fortunately, that many of these rapidly growing string collections are highly repetitive, so that their information content is orders of magnitude lower than their plain size. The statistical compression methods used for classical collections, however, are blind to this repetitiveness, and therefore a new set of techniques has been developed in order to properly exploit it. The resulting indexes form a new generation of data structures able to handle the huge repetitive string collections that we are facing. In this survey we cover the algorithmic developments that have led to these data structures. We describe the distinct compression paradigms that have been used to exploit repetitiveness, the fundamental algorithmic ideas that form the base of all the existing indexes, and the various structures that have been proposed, comparing them both in theoretical and practical aspects. We conclude with the current challenges in this fascinating field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑