用于构建大型隧道化Wheeler图的无前缀解析
Prefix-free parsing for building large tunnelled Wheeler graphs
AI总结:
本研究结合Wheeler图与无前缀解析(PFP)技术,通过对文本解析结果的Wheeler图隧道化再构建原图,解决隧道化最优块选择耗时难题,实现了泛基因组序列集合上更快、更低内存的空间高效索引构建。
摘要:
我们提出了一种为大型重复文本集合(例如包含同一物种多个个体序列的泛基因组数据库)构建空间高效索引的新技术。我们结合了该领域的两项最新技术:Wheeler图(Wheeler graphs,WGs,Gagie等人,2017)和无前缀解析(prefix-free parsing,PFP,Boucher等人,2019)。Wheeler图是一个通用框架,涵盖了多种基于Burrows-Wheeler变换(BWT)的索引,例如FM-index。Wheeler图支持简洁表示,还可以通过采用隧道化思想进一步压缩——该思想利用以并行、等标记路径(称为块)形式存在的冗余,可将这些块合并为单条路径。找到用于隧道化的最优块集合(即可使最终Wheeler图规模最小化的块集合)的问题已被证明是NP完全问题,且仍是隧道化过程中计算难度最大的部分。\n为了在更短时间内找到合适的块集合,我们提出了一种基于无前缀解析(PFP)的新方法。PFP的思路是将输入文本划分为大小大致相等、且有固定字符数重叠的短语。原始文本由短语排名序列(即解析结果)和所有已用短语的列表(即词典)表示。在重复文本中,文本的PFP通常远短于原始文本。为了加快隧道化的块选择过程,我们应用PFP获取文本的解析结果和词典,使用现有启发式算法对解析结果的Wheeler图进行隧道化,随后利用该隧道化的解析结果构建原始文本的紧凑Wheeler图。与不使用PFP、直接从原始文本构建Wheeler图相比,我们的方法在泛基因组序列集合上速度更快、内存占用更少。因此,我们的方法使Wheeler图能够作为泛基因组参考应用于真实数据集。
英文摘要:
We propose a new technique for creating a space-efficient index for large repetitive text collections, such as pangenomic databases containing sequences of many individuals from the same species. We combine two recent techniques from this area: Wheeler graphs (Gagie et al., 2017) and prefix-free parsing (PFP, Boucher et al., 2019). Wheeler graphs (WGs) are a general framework encompassing several indexes based on the Burrows-Wheeler transform (BWT), such as the FM-index. Wheeler graphs admit a succinct representation which can be further compacted by employing the idea of tunnelling, which exploits redundancies in the form of parallel, equally-labelled paths called blocks that can be merged into a single path. The problem of finding the optimal set of blocks for tunnelling, i.e. the one that minimizes the size of the resulting WG, is known to be NP-complete and remains the most computationally challenging part of the tunnelling process. To find an adequate set of blocks in less time, we propose a new method based on the prefix-free parsing (PFP). The idea of PFP is to divide the input text into phrases of roughly equal sizes that overlap by a fixed number of characters. The original text is represented by a sequence of phrase ranks (the parse) and a list of all used phrases (the dictionary). In repetitive texts, the PFP of the text is generally much shorter than the original. To speed up the block selection for tunnelling, we apply the PFP to obtain the parse and the dictionary of the text, tunnel the WG of the parse using existing heuristics and subsequently use this tunnelled parse to construct a compact WG of the original text. Compared with constructing a WG from the original text without PFP, our method is much faster and uses less memory on collections of pangenomic sequences. Therefore, our method enables the use of WGs as a pangenomic reference for real-world datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。