arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1207.3532cs.DScs.DB
收藏

内存有效的De Bruijn图构造

Memory Efficient De Bruijn Graph Construction

Yang Li, Pegah Kamousi, Fangqiu Han, Shengqi Yang, Xifeng Yan, Subhash Suri

摘要:

大规模并行DNA测序技术正在彻底改变基因组学研究。以低成本产生的数十亿个短读段可以被组装用于重建整个基因组。不幸的是,现有从头组装算法的大内存占用使得为哺乳动物等高等真核生物完成组装具有挑战性。在这项工作中,我们研究了构建de Bruijn图的内存问题,这是领先的组装算法中的核心任务,对于大型基因组,通常会消耗数百GB的内存。我们提出了一种基于磁盘的分区方法,称为最小子串分区(MSP),使用不到10千兆字节的内存完成任务,没有运行时减速。MSP将短读分为多个小的不相交分区,以便每个分区可以加载到内存中,单独处理,然后与其他分区合并以形成de Bruijn图。通过利用k-mer(长度为k的子串)之间的重叠,MSP实现了惊人的压缩比:分区的总大小从$Θ(kn)$减少到$Θ(n)$,其中$n$是短读段数据库的大小,而$k$是$k$ -mer的长度。实验结果表明,我们的方法可以建立de Bruijn图使用商品计算机的任何大容量的序列数据集。

英文摘要:

Massively parallel DNA sequencing technologies are revolutionizing genomics research. Billions of short reads generated at low costs can be assembled for reconstructing the whole genomes. Unfortunately, the large memory footprint of the existing de novo assembly algorithms makes it challenging to get the assembly done for higher eukaryotes like mammals. In this work, we investigate the memory issue of constructing de Bruijn graph, a core task in leading assembly algorithms, which often consumes several hundreds of gigabytes memory for large genomes. We propose a disk-based partition method, called Minimum Substring Partitioning (MSP), to complete the task using less than 10 gigabytes memory, without runtime slowdown. MSP breaks the short reads into multiple small disjoint partitions so that each partition can be loaded into memory, processed individually and later merged with others to form a de Bruijn graph. By leveraging the overlaps among the k-mers (substring of length k), MSP achieves astonishing compression ratio: The total size of partitions is reduced from $Θ(kn)$ to $Θ(n)$, where $n$ is the size of the short read database, and $k$ is the length of a $k$-mer. Experimental results show that our method can build de Bruijn graphs using a commodity computer for any large-volume sequence dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑