arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.03151cs.SEcs.LG
收藏

基于 DQN 的半监督学习用于日志异常检测

Semi-supervised learning via DQN for log anomaly detection

Huazhong University of Science and Technology(华中科技大学)

Yingying He, Xiaobing Pei

AI总结:

提出半监督日志异常检测方法 DQNLog,通过深度强化学习结合少量标记与大规模未标记数据,设计偏向异常的状态转移函数与联合奖励函数,有效解决了数据不平衡与误报漏报高的问题。

摘要:

日志异常检测是现代软件系统安全和维护的关键组成部分,为系统监控、运维和故障排查提供重要的支撑和依据。它帮助运维人员及时识别和解决问题。然而,当前的日志异常检测方法仍面临诸多挑战,如未充分利用未标记数据、正常与异常类数据不平衡以及高假阳性和假阴性率,导致异常识别效果不佳。在本研究中,我们提出了一种名为 DQNLog 的半监督日志异常检测方法,该方法集成了深度强化学习,通过利用少量标记数据和大规模未标记数据来增强异常检测性能。针对数据不平衡和标记不足的问题,我们设计了一种基于余弦相似度且偏向异常的状态转移函数,旨在捕获语义相似的异常,而非偏向多数类。为了增强模型学习异常的能力,我们设计了一个联合奖励函数,鼓励模型利用已标记的异常并探索未标记的异常,从而降低假阳性和假阴性。此外,为防止模型因错误估计而偏离正常轨迹,我们在损失函数中引入正则化项,以确保模型在更新过程中保留先验知识。我们在三个广泛使用的数据集上对 DQNLog 进行了评估,结果表明其能够有效利用大规模未标记数据,并在所有实验数据集上取得有希望的结果。

英文摘要:

Log anomaly detection is a critical component in modern software system security and maintenance, serving as a crucial support and basis for system monitoring, operation, and troubleshooting. It aids operations personnel in timely identification and resolution of issues. However, current methods in log anomaly detection still face challenges such as underutilization of unlabeled data, imbalance between normal and anomaly class data, and high rates of false positives and false negatives, leading to insufficient effectiveness in anomaly recognition. In this study, we propose a semi-supervised log anomaly detection method named DQNLog, which integrates deep reinforcement learning to enhance anomaly detection performance by leveraging a small amount of labeled data and large-scale unlabeled data. To address issues of imbalanced data and insufficient labeling, we design a state transition function biased towards anomalies based on cosine similarity, aiming to capture semantic-similar anomalies rather than favoring the majority class. To enhance the model's capability in learning anomalies, we devise a joint reward function that encourages the model to utilize labeled anomalies and explore unlabeled anomalies, thereby reducing false positives and false negatives. Additionally, to prevent the model from deviating from normal trajectories due to misestimation, we introduce a regularization term in the loss function to ensure the model retains prior knowledge during updates. We evaluate DQNLog on three widely used datasets, demonstrating its ability to effectively utilize large-scale unlabeled data and achieve promising results across all experimental datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑