arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1810.06012astro-ph.IM
收藏

大容量脉冲星数据流处理流水线

A Processing Pipeline for High Volume Pulsar Data Streams

R. J. Lyon, B. W. Stappers, L. Levin, M. B. Mickaliger, A. Scaife

摘要:

脉冲星数据分析管道历来由定制的软件系统组成,支持数据的离线分析。然而,现代数据采集系统正在使离线分析变得不切实际。它们通常会同时输出多个大容量数据流,从而显著提高数据捕获率。这导致大量数据的积累,而这些数据的保留成本高得令人望而却步。当离线分析由于成本原因变得不可行时,为了保持处理能力,需要转向在线数据处理。本文提出了四个贡献,促进这一转变方面的搜索无线电信标:i)它描述了现代脉冲星搜索科学的关键组成部分(非信号处理)流水线,ii)它检查通过现有工具实现在线脉冲星搜索的可行性,iii)识别并解释了阻止容易过渡到在线搜索的问题,以及最后iv)它提供了能够克服这些问题的新的原型管道的设计。使用商用现成(COTS)软件组件实现,可部署系统是开源的,简单的,可扩展的,生产成本低。它有可能实现脉冲星搜索平方公里阵列(SKA)的设计要求,通过模拟SKA负载下的测试说明。

英文摘要:

Pulsar data analysis pipelines have historically been comprised of bespoke software systems, supporting the off-line analysis of data. However modern data acquisition systems are making off-line analyses impractical. They often output multiple simultaneous high volume data streams, significantly increasing data capture rates. This leads to the accumulation of large data volumes, which are prohibitively expensive to retain. To maintain processing capabilities when off-line analysis becomes infeasible due to cost, requires a shift to on-line data processing. This paper makes four contributions facilitating this shift with respect to the search for radio pulsars: i) it characterises for the modern era, the key components of a pulsar search science (not signal processing) pipeline, ii) it examines the feasibility of implementing on-line pulsar search via existing tools, iii) problems preventing an easy transition to on-line search are identified and explained, and finally iv) it provides the design for a new prototype pipeline capable of overcoming such problems. Realised using Commercial off-the-shelf (COTS) software components, the deployable system is open source, simple, scalable, and cheap to produce. It has the potential to achieve pulsar search design requirements for the Square Kilometre Array (SKA), illustrated via testing under simulated SKA loads.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑