通过减少事件增强基于日志的异常检测模型：一项实证研究
Reducing Events to Augment Log-based Anomaly Detection Models: An Empirical Study
AI总结：
本研究针对现有基于日志的异常检测方法依赖海量日志、效率低且易受噪声干扰的问题，通过实证分析日志数量与异常表征效果的关联，提出LogCleaner日志精简方法，可减少超70%日志、提速约300%并提升检测性能。
摘要：
随着软件系统日益复杂，精准检测异常已变得既必要又具挑战性。当前基于日志的异常检测方法严重依赖海量日志数据，导致推理效率低下，还可能受噪声日志误导。然而，日志精简对异常检测效果的量化影响尚未得到探究。为此，我们首先在三个数据集上对六种不同模型开展了综合研究。通过该研究，我们量化了日志数量的影响及其表征异常的有效性，发现了三类对模型性能影响各异的独特日志事件类型。基于这些发现，我们提出LogCleaner：一种用于异常检测场景下自动精简日志事件的高效方法。作为软件系统与模型之间的中间件，LogCleaner会持续更新并过滤原始生成日志中的反事件与重复事件。实验结果表明，LogCleaner能够在异常检测中减少超过70%的日志事件，将模型推理速度提升约300%，并普遍提升异常检测模型的性能。
英文摘要：
As software systems grow increasingly intricate, the precise detection of anomalies have become both essential and challenging. Current log-based anomaly detection methods depend heavily on vast amounts of log data leading to inefficient inference and potential misguidance by noise logs. However, the quantitative effects of log reduction on the effectiveness of anomaly detection remain unexplored. Therefore, we first conduct a comprehensive study on six distinct models spanning three datasets. Through the study, the impact of log quantity and their effectiveness in representing anomalies is qualifies, uncovering three distinctive log event types that differently influence model performance. Drawing from these insights, we propose LogCleaner: an efficient methodology for the automatic reduction of log events in the context of anomaly detection. Serving as middleware between software systems and models, LogCleaner continuously updates and filters anti-events and duplicative-events in the raw generated logs. Experimental outcomes highlight LogCleaner's capability to reduce over 70% of log events in anomaly detection, accelerating the model's inference speed by approximately 300%, and universally improving the performance of models for anomaly detection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
