在双通道噪声数据中高效搜索小信号：大数据观测中的一项挑战
Efficient searches of small signals across two-channel noisy data: a challenge in Big Data observations
浏览论文内容
中文总结 AI 辅助
针对大数据观测中搜索小信号的效率挑战，本文提出超额概率比（EPR）方法，其性能与皮尔逊系数相当，可在基本不损失灵敏度的前提下提升效率。
中文摘要 AI 辅助
在噪声数据中搜索新的小信号，更适合通过两个或更多独立运行通道的相关性来开展。潜在感兴趣的信号存在于超过κσ的尾部区域，其中κ表示数据标准差σ的倍数。由于在大数据分析及更广泛的异构计算中，数据移动是主要成本因素，通过将相关性计算限制在双通道数据中超过κσ的尾部区域，可优化效率。当κ≥2时，已能实现至少一个数量级的数据缩减。本文采用一种新的“超额概率比（Excess Probability Ratio，EPR）”研究该方法，该方法将来自超过κσ阈值的尾部产生的布尔数据进行关联。我们将EPR的性能与传统的直接互相关（Direct Cross-Correlation，DCC）和适用于无阈值原始数据的皮尔逊系数（Pearson Coefficient，PC）进行比较和排序。该基准测试在不同的背景噪声组合（高斯、泊松和均匀分布）及信号类型（高斯、泊松、均匀、啁啾和正弦波）上开展。结果显示，EPR的性能与PC相当，为效率的显著提升提供了新方法，且基本无灵敏度损失，适用于当前大数据天文台的时代。
英文摘要
Searching for novel small signals in noisy data is preferably pursued by correlation in two or more independently operating channels. Signals of potential interest exist in tails beyond $κσ$, where $κ$ denotes a multiple of the standard deviation $σ$ of the data. Since moving data is a major cost factor in Big Data analysis and heterogeneous computing more generally, efficiency may be optimized by restricting the correlations computation to the tails of two-channel data exceeding $κσ$. Already, a moderate value $κ\gtrsim2$ realizes a data-reduction by at least an order of magnitude. Here, we study this approach using a novel {\it Excess Probability Ratio} (EPR), correlating Boolean data resulting from tails beyond a cut-off $κσ$. We compare and rank EPR performance against conventional direct cross-correlation (DCC) and Pearson coefficient (PC), applicable to the original data with no cut-off. This benchmark is performed over different combinations of background noise Gaussian, Poisson and Uniform and signals Gaussian, Poisson, Uniform, Chirps and Sine waves. Results show performance of EPR to be comparable to that of PC, providing a new approach for significant improvements in efficiency with essentially no loss of sensitivity, relevant to the present era of Big Data observatories.
发表机构
- Sejong University（世宗大学）
- INAF-OAS Bologna（意大利国家天体物理研究所博洛尼亚天文台）
机构由 AI 辅助整理，请以论文原文为准。
