arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.20503cs.SEcs.AIcs.LG
收藏

哪些信息有助于基于日志的异常检测?来自一种可配置Transformer方法的见解

What Information Contributes to Log-based Anomaly Detection? Insights from a Configurable Transformer-Based Approach

Xingfang Wu, Heng Li, Foutse Khomh

AI总结:

本文提出一种可配置的Transformer日志异常检测模型,通过不同长度日志序列和特征组合实验,发现事件发生信息对异常识别最关键,而序列与时间信息影响有限,并指出需构建更复杂的新数据集。

摘要:

日志数据由源代码中的日志语句生成,能够提供软件应用和系统执行过程的洞察。当前最先进的基于日志的异常检测方法通常利用深度学习模型来捕获日志数据中的语义或序列信息,并检测异常的运行时行为。然而,这些不同类型信息的影响尚不明确。此外,大多数现有方法忽略了日志数据中的时间戳,而时间戳可能提供细粒度的序列和时间信息。在本工作中,我们提出了一种可配置的基于Transformer的异常检测模型,该模型能够捕获日志数据中的语义、序列和时间信息,并允许我们将不同类型的信息配置为模型特征。此外,我们使用不同长度的日志序列训练和评估所提出的模型,从而克服了现有方法依赖固定长度或时间窗口日志序列作为输入的约束。借助所提出的模型,我们开展了一系列使用不同输入特征组合的实验,以评估不同类型信息在异常检测中的作用。与基线方法相比,该模型在输入不同长度日志序列时能够获得具有竞争力且持续稳定的性能。结果表明,事件发生信息在识别异常中起关键作用,而序列和时间信息对所研究的公开数据集上的异常检测影响不显著。另一方面,研究结果也揭示了所研究公开数据集的简单性,并强调了构建包含不同类型异常的新数据集以更好地评估异常检测模型性能的重要性。

英文摘要:

Log data are generated from logging statements in the source code, providing insights into the execution processes of software applications and systems. State-of-the-art log-based anomaly detection approaches typically leverage deep learning models to capture the semantic or sequential information in the log data and detect anomalous runtime behaviors. However, the impacts of these different types of information are not clear. In addition, most existing approaches ignore the timestamps in log data, which can potentially provide fine-grained sequential and temporal information. In this work, we propose a configurable Transformer-based anomaly detection model that can capture the semantic, sequential, and temporal information in the log data and allows us to configure the different types of information as the model's features. Additionally, we train and evaluate the proposed model using log sequences of different lengths, thus overcoming the constraint of existing methods that rely on fixed-length or time-windowed log sequences as inputs. With the proposed model, we conduct a series of experiments with different combinations of input features to evaluate the roles of different types of information in anomaly detection. The model can attain competitive and consistently stable performance compared to the baselines when presented with log sequences of varying lengths. The results indicate that the event occurrence information plays a key role in identifying anomalies, while the impact of the sequential and temporal information is not significant for anomaly detection on the studied public datasets. On the other hand, the findings also reveal the simplicity of the studied public datasets and highlight the importance of constructing new datasets that contain different types of anomalies to better evaluate the performance of anomaly detection models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑