通过以存储为中心的系统设计实现快速、高效和低成本的基因组和宏基因组分析
Enabling Fast, Efficient, and Low-Cost Genomic and Metagenomic Analyses via Storage-Centric System Designs
AI总结:
针对基因组和宏基因组序列数据分析与存储挑战,提出以存储为中心的定制系统,包括GenStore、MegIS、GRAINS和SAGe,可在存储内分析数据、高效存储与访问,提升性能、能效和成本效益。
摘要:
由于分析和存储大量基因组和宏基因组序列数据面临挑战,虽已努力加速分析和压缩存储数据,但在访问存储序列数据并提供给分析单元方面仍有两大问题。本文提出定制的以存储为中心的系统,可在存储内分析(元)基因组数据,实现大规模序列数据的高压缩存储和高性能访问,减轻数据移动、计算和数据准备的开销。通过GenStore、MegIS、GRAINS和SAGe展示了这些系统显著提高了性能、能源效率和成本效益,希望促进(元)基因组学的更广泛应用并激发其他健康和生命科学数据密集型领域的研究。
英文摘要:
Due to the challenges of analyzing and storing massive volumes of genomic and metagenomic sequence data, significant efforts have been made to accelerate (meta)genomic analyses and store sequence data compressed. Despite the benefits of these techniques, we identify two major outstanding problems in accessing stored sequence data and supplying it to the analysis units: (i) the data movement bottleneck due to moving large amounts of low-reuse data from storage and the unnecessary burden on the rest of the system, and (ii) the data preparation bottleneck, where compressed sequence data needs to be first decompressed and formatted before analysis. We present customized storage-centric systems, which efficiently (i) analyze (meta)genomic data inside storage, and (ii) enable highly-compressed storage and high-performance access of large-scale sequence data, thereby alleviating the overheads of data movement, computation, and data preparation. First, we introduce GenStore, an in-storage processing system that filters genomic data not requiring expensive computation directly inside storage. Second, we propose MegIS, an in-storage processing system that significantly reduces the data movement overhead of metagenomic analysis. Third, we introduce GRAINS, a storage-centric system for analysis on large-scale (meta)genomic graphs in storage. Fourth, we propose SAGe, an algorithm-architecture co-design for highly-compressed storage and high-performance access of sequence data. We demonstrate that the proposed systems significantly (e.g., by one to two orders of magnitude) improve performance, energy efficiency, and cost-efficiency, all at the same time. We hope these systems facilitate broader adoption of (meta)genomics and inspire research on other data-intensive domains in health and life sciences.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。