arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.07660cs.LG
收藏

大规模运营网络时间序列数据上异常事件预测的半监督方法

A Semi-Supervised Approach for Abnormal Event Prediction on Large Operational Network Time-Series Data

Department of Computer Science and Engineering(计算机科学与工程系);University of Minnesota, Twin Cities(明尼苏达大学,双城校区)

Yijun Lin, Yao-Yi Chiang

AI总结:

本文提出一种半监督方法,通过捕获网络时间序列间及跨时间点的依赖关系生成表示,利用有限标记数据和未标记数据学习可分离嵌入空间,在大型真实网络日志上显著优于现有异常事件检测方法。

摘要:

大型网络日志记录了网络中异构设备和传感器生成的多变量时间序列,通常能揭示异常活动的重要信息,如网络入侵和设备故障。现有的用于多变量时间序列异常检测的机器学习方法通常假设:1)正常序列在训练无监督模型时具有一致的行为,或2)需要大量标记的正常和异常序列用于监督模型。然而,在实践中,正常网络活动可能表现出显著变化的序列模式(例如,在部分网络流量重新路由之前和之后)。此外,记录的异常事件可能很稀疏。本文提出了一种新颖的半监督方法,该方法能有效捕获网络时间序列之间以及跨时间点的依赖关系,以生成有意义的网络活动表示,用于预测异常事件。该方法可以利用有限的标记数据显式学习正常和异常样本的可分离嵌入空间,并有效利用未标记数据来应对训练数据稀缺问题。实验表明,我们的方法在大型真实世界网络日志上的事件检测性能显著优于最先进的方法。

英文摘要:

Large network logs, recording multivariate time series generated from heterogeneous devices and sensors in a network, can often reveal important information about abnormal activities, such as network intrusions and device malfunctions. Existing machine learning methods for anomaly detection on multivariate time series typically assume that 1) normal sequences would have consistent behavior for training unsupervised models, or 2) require a large set of labeled normal and abnormal sequences for supervised models. However, in practice, normal network activities can demonstrate significantly varying sequence patterns (e.g., before and after rerouting partial network traffic). Also, the recorded abnormal events can be sparse. This paper presents a novel semi-supervised method that efficiently captures dependencies between network time series and across time points to generate meaningful representations of network activities for predicting abnormal events. The method can use the limited labeled data to explicitly learn separable embedding space for normal and abnormal samples and effectively leverage unlabeled data to handle training data scarcity. The experiments demonstrate that our approach significantly outperformed state-of-the-art approaches for event detection on a large real-world network log.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑