arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.06175cs.SEcs.AIcs.LG
收藏

MTAD:面向多变量时间序列异常检测的工具与基准

MTAD: Tools and Benchmarks for Multivariate Time Series Anomaly Detection

The Chinese University of Hong Kong(香港中文大学);Sun Yat-sen University(中山大学)

Jinyang Liu, Wenwei Gu, Zhuangbin Chen, Yichen Li, Yuxin Su, Michael R. Lyu

AI总结:

本文系统评估了十二种多变量KPI异常检测方法,提出新指标salience,并发布统一工具包与基准结果,为工业部署和后续研究提供基础。

摘要:

关键性能指标(KPI)是保障众多软件系统可靠性与稳定性的重要时间序列度量。它们忠实地记录运行时状态,以帮助理解系统异常行为,并为工程师定位根本原因提供有价值的线索。然而,现代软件系统前所未有的规模和复杂性使得KPI数据量急剧膨胀。因此,许多传统的KPI异常检测方法变得不再实用,这推动了学术界和工业界基于机器学习解决方案的快速发展。然而,目前这些KPI异常检测方法之间缺乏严格的比较,重新实现这些方法也需要付出不小的努力。此外,我们观察到不同工作采用独立的评估流程和不同的评估指标。其中一些可能无法充分揭示模型的能力,而另一些则可能制造出进展的假象。为了更好地理解不同KPI异常检测器的特性并解决评估问题,本文对十二种最先进的方法进行了全面的回顾与评估,并提出了一种名为salience的新指标。具体而言,所选方法包括五种基于传统机器学习的方法和七种基于深度学习的方法。这些方法在五个公开可用的多变量KPI数据集上进行了评估。我们还发布了一个具有易用接口的统一工具包。我们从准确率、salience、效率和延迟等方面报告了基准测试结果,这些指标对于工业部署具有实际重要意义。我们相信,我们的工作可以作为未来学术研究和工业应用的基础。

英文摘要:

Key Performance Indicators (KPIs) are essential time-series metrics for ensuring the reliability and stability of many software systems. They faithfully record runtime states to facilitate the understanding of anomalous system behaviors and provide informative clues for engineers to pinpoint the root causes. The unprecedented scale and complexity of modern software systems, however, make the volume of KPIs explode. Consequently, many traditional methods of KPI anomaly detection become impractical, which serves as a catalyst for the fast development of machine learning-based solutions in both academia and industry. However, there is currently a lack of rigorous comparison among these KPI anomaly detection methods, and re-implementation demands a non-trivial effort. Moreover, we observe that different works adopt independent evaluation processes with different metrics. Some of them may not fully reveal the capability of a model and some are creating an illusion of progress. To better understand the characteristics of different KPI anomaly detectors and address the evaluation issue, in this paper, we provide a comprehensive review and evaluation of twelve state-of-the-art methods, and propose a novel metric called salience. Particularly, the selected methods include five traditional machine learning-based methods and seven deep learning-based methods. These methods are evaluated with five multivariate KPI datasets that are publicly available. A unified toolkit with easy-to-use interfaces is also released. We report the benchmark results in terms of accuracy, salience, efficiency, and delay, which are of practical importance for industrial deployment. We believe our work can contribute as a basis for future academic research and industrial application.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑