arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.09565cs.DCq-bio.GN
收藏

通过改进内存访问将关键生物信息学任务加速100倍

Accelerating key bioinformatics tasks 100-fold by improving memory access

Igor Sfiligoi, Daniel McDonald, Rob Knight

摘要:

大多数实验科学现在都依赖于计算,生物科学也不例外。随着数据集变得越来越大,计算成本也随之增加,这使得科学家使用的代码的适当优化变得越来越重要。近年来开发的许多代码都基于基于Python的NumPy,因为它易于使用和良好的性能特性。然而,NumPy的可组合性通常不能很好地与现代CPU的多层性质相匹配,使得任何非平凡的多步算法都受到外部存储器访问速度的限制,这比CPU的计算能力慢数百倍。为了充分利用CPU的计算能力,必须保持工作内存占用足够小,以适应CPU缓存,这需要将问题分成更小的部分,并尽可能多地融合在一起。在本文中,我们基于这些原则对scikit-bio库中的两个重要功能进行了更改,即主坐标分析和Mantel测试,这些广泛使用的通用工具的速度提高了100倍以上。

英文摘要:

Most experimental sciences now rely on computing, and biological sciences are no exception. As datasets get bigger, so do the computing costs, making proper optimization of the codes used by scientists increasingly important. Many of the codes developed in recent years are based on the Python-based NumPy, due to its ease of use and good performance characteristics. The composable nature of NumPy, however, does not generally play well with the multi-tier nature of modern CPUs, making any non-trivial multi-step algorithm limited by the external memory access speeds, which are hundreds of times slower than the CPU's compute capabilities. In order to fully utilize the CPU compute capabilities, one must keep the working memory footprint small enough to fit in the CPU caches, which requires splitting the problem into smaller portions and fusing together as many steps as possible. In this paper, we present changes based on these principles to two important functions in the scikit-bio library, principal coordinates analysis and the Mantel test, that resulted in over 100x speed improvement in these widely used, general-purpose tools.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑