arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2003.06222stat.MLcs.LGstat.ME
收藏

变点检测算法的评价

An Evaluation of Change Point Detection Algorithms

The University of Edinburgh(爱丁堡大学);The Alan Turing Institute(艾伦·图灵研究所)

Gerrit J. J. van den Burg, Christopher K. I. Williams

摘要:

变化点检测是时间序列分析的重要组成部分,因为变化点的存在指示数据生成过程中的突然且显著的变化。虽然已经提出了许多算法的变化点检测,相对较少的注意力已经支付给评估其性能的现实世界的时间序列。算法通常是在模拟数据和少量常用的不可靠的地面真相系列上进行评估的。显然,这并没有提供足够的洞察力,这些算法的比较性能。因此,与其开发另一种变点检测方法,我们认为更重要的是在真实数据上正确评估现有算法。为了实现这一目标,我们提出了一个数据集,专门设计的变化点检测算法,包括37个时间序列从不同的应用领域的评估。每个系列都由五名人类注释员进行注释,以提供关于变化点的存在和位置的基本事实。我们分析了人类注释的一致性,并描述了评估指标,可用于测量算法的性能,在存在多个地面实况注释。接下来,我们提出了一个基准研究,其中14个算法进行评估的数据集中的每个时间序列。我们的目标是,这个数据集将作为一个试验场,在新的变化点检测算法的发展。

英文摘要:

Change point detection is an important part of time series analysis, as the presence of a change point indicates an abrupt and significant change in the data generating process. While many algorithms for change point detection have been proposed, comparatively little attention has been paid to evaluating their performance on real-world time series. Algorithms are typically evaluated on simulated data and a small number of commonly-used series with unreliable ground truth. Clearly this does not provide sufficient insight into the comparative performance of these algorithms. Therefore, instead of developing yet another change point detection method, we consider it vastly more important to properly evaluate existing algorithms on real-world data. To achieve this, we present a data set specifically designed for the evaluation of change point detection algorithms that consists of 37 time series from various application domains. Each series was annotated by five human annotators to provide ground truth on the presence and location of change points. We analyze the consistency of the human annotators, and describe evaluation metrics that can be used to measure algorithm performance in the presence of multiple ground truth annotations. Next, we present a benchmark study where 14 algorithms are evaluated on each of the time series in the data set. Our aim is that this data set will serve as a proving ground in the development of novel change point detection algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑