Modern software systems produce vast amounts of logs, serving as an essential resource for anomaly detection. Artificial Intelligence for IT Operations (AIOps) tools have been developed to automate the process of log-based anomaly detection for software systems. Three practical challenges are widely recognized in this field: high data labeling costs, evolving logs in dynamic systems, and adaptability across different systems. In this paper, we propose CroSysLog, an AIOps tool for log-event level anomaly detection, specifically designed in response to these challenges. Following prior approaches, CroSysLog uses a neural representation approach to gain a nuanced understanding of logs and generate representations for individual log events accordingly. CroSysLog can be trained on source systems with sufficient labeled logs from open datasets to achieve robustness, and then efficiently adapt to target systems with a few labeled log events for effective anomaly detection. We evaluate CroSysLog using open datasets of four large-scale distributed supercomputing systems: BGL, Thunderbird, Liberty, and Spirit. We used random log splits, maintaining the chronological order of consecutive log events, from these systems to train and evaluate CroSysLog. These splits were widely distributed across a one/two-year span of each system's log collection duration, thereby capturing the evolving nature of the logs in each system. Our results show that, after training CroSysLog on Liberty and BGL as source systems, CroSysLog can efficiently adapt to target systems Thunderbird and Spirit using a few labeled log events from each target system, effectively performing anomaly detection for these target systems. The results demonstrate that CroSysLog is a practical, scalable, and adaptable tool for log-event level anomaly detection in operational and maintenance contexts of software systems.
Architecting software monitors for control-flow anomaly detection through large language models and conformance checking
University of Naples Federico II(那不勒斯费德里科二世大学);University of Applied Sciences and Arts of Southern Switzerland(瑞士南部应用科学与艺术大学);University of Florence(佛罗伦萨大学);Linnaeus University(林奈大学)
Pinpointing Anomaly Events in Logs from Stability Testing -- N-Grams vs. Deep-Learning
由于稳定性测试执行日志可能很长,软件工程师需要帮助定位异常事件。我们开发并评估了两个模型,用于对单个日志事件进行评分,即N-Gram模型和具有LSTM(长短期记忆)的深度学习模型。两者都只在正常的日志序列上训练。我们评估的模型与长日志序列的Android稳定性测试在我们公司的情况下,和短日志序列的HDFS(Hadoop分布式文件系统)的公共数据集。我们评估下一个事件的预测精度和计算效率。LSTM模型在稳定性测试日志中更准确(0.848 vs 0.865),而在HDFS日志中,N-Gram稍微更准确(0.904 vs 0.900)。与Deep模型相比,N-Gram模型具有更高的计算效率(4到13秒vs 16分钟到近4小时),使其成为我们案例公司的首选。对单个日志事件的错误性进行评分似乎是对失败测试用例的根本原因分析的一个很好的帮助,我们的案例公司计划将其添加到其在线服务中。尽管最近在软件系统异常检测中使用深度学习的浪潮激增,但我们发现这样做的好处有限。然而,未来的工作应该考虑我们的发现是否适用于不同的LSTM模型超参数,其他数据集,以及其他深度学习方法,这些方法比基于LSTM的模型具有更好的准确性和计算效率。