arXiv 1406.43762014-06-17q-bio.GN

BEETL-fastq：用于DNA读取的可搜索压缩存档

BEETL-fastq: a searchable compressed archive for DNA reads

Lilian Janin, Ole Schulz-Trieglaff, Anthony J. Cox

备注：: As submitted to Bioinformatics. Bioinformatics should be considered the original place of publication of this work, please cite accordingly

摘要：

动机：FASTQ是DNA测序数据的标准文件格式，存储核苷酸和质量评分。一个典型的测序研究可以很容易地生成数百GB的FASTQ文件，而ENA和NCBI等公共档案以及癌症基因组图谱等大型国际合作可以以这种格式积累许多TB的数据。文本压缩工具，如gzip，通常用于减少存储负担，但有一个缺点，即数据必须在使用前解压缩。在这里，我们提出BEETL-fastq，一个工具，不仅压缩FASTQ格式的DNA读比gzip更复杂，但也允许快速搜索归档序列内的$k$聚体查询。重要的是，每个匹配的读段或读段对的完整FASTQ记录被返回，允许搜索结果被直接输送到接受FASTQ数据作为输入的许多标准工具中的任何一个。结果如下：我们表明，6.6 TB的人类读取FASTQ格式可以转换为1.7 TB的索引文件，从那里我们可以搜索1，10，100，1000，100万，分别在3，8，14，45和567秒加上20毫秒的30-mer的输出读取。搜索能力的有用的应用突出显示，包括结构变异断点的基因分型和“在硅片下拉”实验中，只有覆盖感兴趣的区域的读数被选择性地提取用于变异检测或可视化的目的。可用性：BEETL-fastq是BEETL库的一部分，可在git@github.com上作为github存储库获得：BEETL/BEETL.git。

英文摘要：

Motivation: FASTQ is a standard file format for DNA sequencing data which stores both nucleotides and quality scores. A typical sequencing study can easily generate hundreds of gigabytes of FASTQ files, while public archives such as ENA and NCBI and large international collaborations such as the Cancer Genome Atlas can accumulate many terabytes of data in this format. Text compression tools such as gzip are often employed to reduce the storage burden, but have the disadvantage that the data must be decompressed before it can be used. Here we present BEETL-fastq, a tool that not only compresses FASTQ-formatted DNA reads more compactly than gzip, but also permits rapid search for $k$-mer queries within the archived sequences. Importantly, the full FASTQ record of each matching read or read pair is returned, allowing the search results to be piped directly to any of the many standard tools that accept FASTQ data as input. Results: We show that 6.6 terabytes of human reads in FASTQ format can be transformed into 1.7 terabytes of indexed files, from where we can search for 1, 10, 100, 1000, a million of 30-mers in respectively 3, 8, 14, 45 and 567 seconds plus 20 ms per output read. Useful applications of the search capability are highlighted, including the genotyping of structural variant breakpoints and "in silico pull-down" experiments in which only the reads that cover a region of interest are selectively extracted for the purposes of variant calling or visualization. Availability: BEETL-fastq is part of the BEETL library, available as a github repository at git@github.com:BEETL/BEETL.git.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。