arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.11570cs.AIcs.CLcs.SE
收藏

基于预训练语言模型的鲁棒可移植日志异常检测

Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models

TU Berlin(柏林工业大学)

Harold Ott, Jasmin Bogatinovski, Alexander Acker, Sasho Nedelkoski, Odej Kao

摘要:

诸如云的大型计算机系统中的异常或故障对通信、计算和存储信息的大量用户具有影响。因此,及时准确的异常检测对于这些日益重要的系统的可靠性、安全性、安全操作和减轻损失是必要的。最近,软件行业的发展提出了几个需要解决的问题,包括(1)解决由于软件升级引起的软件发展,以及(2)解决冷启动问题,其中来自感兴趣的系统的数据不可用。在本文中,我们提出了一个框架,异常检测日志数据,作为一个主要的故障排除源的系统信息。为此,我们利用预训练的通用语言模型来保留日志消息的语义,并将其映射到日志向量嵌入。关键的想法是,这些表示的日志是强大的,不太不变的日志中的变化,因此,在一个更好的泛化的异常检测模型的结果。我们在云数据集上进行了几个实验,评估不同的语言模型,以获得数字日志表示,如BERT,GPT-2和XL。鲁棒性通过逐渐改变日志消息来评估,以模拟语义的变化。我们的研究结果表明,所提出的方法实现了高性能和鲁棒性,这开辟了未来的研究在这个方向上的可能性。

英文摘要:

Anomalies or failures in large computer systems, such as the cloud, have an impact on a large number of users that communicate, compute, and store information. Therefore, timely and accurate anomaly detection is necessary for reliability, security, safe operation, and mitigation of losses in these increasingly important systems. Recently, the evolution of the software industry opens up several problems that need to be tackled including (1) addressing the software evolution due software upgrades, and (2) solving the cold-start problem, where data from the system of interest is not available. In this paper, we propose a framework for anomaly detection in log data, as a major troubleshooting source of system information. To that end, we utilize pre-trained general-purpose language models to preserve the semantics of log messages and map them into log vector embeddings. The key idea is that these representations for the logs are robust and less invariant to changes in the logs, and therefore, result in a better generalization of the anomaly detection models. We perform several experiments on a cloud dataset evaluating different language models for obtaining numerical log representations such as BERT, GPT-2, and XL. The robustness is evaluated by gradually altering log messages, to simulate a change in semantics. Our results show that the proposed approach achieves high performance and robustness, which opens up possibilities for future research in this direction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑