LLM 遇见 ML:不稳定日志上的数据高效异常检测
LLM meets ML: Data-efficient Anomaly Detection on Unstable Logs
AI总结:
针对不稳定日志异常检测数据不足问题,提出结合ML模型与LLM的混合方法FlexLog,通过集成学习、缓存和RAG,在减少标注数据的同时提升F1分数并保持高效推理。
摘要:
大多数基于日志的异常检测器假设日志是稳定的,尽管由于软件或环境变化,日志常常是不稳定的。因此,在不稳定日志上的异常检测(ULAD)是一个更现实但研究不足的挑战。当前方法主要采用机器学习(ML)模型,这些模型通常需要大量标注数据进行训练。为了缓解数据不足的问题,我们提出了 FlexLog,一种用于 ULAD 的新型混合方法,它通过集成学习将 ML 模型——决策树、k 近邻和前馈神经网络——与大语言模型(Mistral)相结合。FlexLog 还结合了缓存和检索增强生成(RAG)以进一步提高效率和有效性。为了评估 FlexLog,我们为任务配置了四个数据集,即 ADFA-U、LOGEVOL-U、SynHDFS-U 和 SYNEVOL-U。FlexLog 在 F1 分数上至少比所有基线高出 1.2 个百分点(pp),同时使用的标注数据少得多(减少 62.87 pp)。在与基线相同数据量下训练时,在不同训练数据集大小下,FlexLog 在 ADFA-U 上的 F1 分数最高提升 13 pp。此外,FlexLog 保持每个日志序列的推理时间低于一秒,使其适用于大多数应用,但延迟敏感系统除外。进一步分析揭示了 FlexLog 关键组件(缓存、RAG 和集成学习)的积极影响。
英文摘要:
Most log-based anomaly detectors assume logs are stable, though logs are often unstable due to software or environmental changes. Anomaly detection on unstable logs (ULAD) is therefore a more realistic, yet under-investigated challenge. Current approaches predominantly employ machine learning (ML) models, which often require extensive labeled data for training. To mitigate data insufficiency, we propose FlexLog, a novel hybrid approach for ULAD that combines ML models -- decision tree, k-nearest neighbors, and a feedforward neural network -- with a Large Language Model (Mistral) through ensemble learning. FlexLog also incorporates a cache and retrieval-augmented generation (RAG) to further enhance efficiency and effectiveness. To evaluate FlexLog, we configured four datasets for \task, namely ADFA-U, LOGEVOL-U, SynHDFS-U, and SYNEVOL-U. FlexLog outperforms all baselines by at least 1.2 percentage points (pp) in F1 score while using much less labeled data (62.87 pp reduction). When trained on the same amount of data as the baselines, FlexLog achieves up to a 13 pp increase in F1 score on ADFA-U across varying training dataset sizes. Additionally, FlexLog maintains inference time under one second per log sequence, making it suitable for most applications, except latency-sensitive systems. Further analysis reveals the positive impact of FlexLog's key components: cache, RAG and ensemble learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。