arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.04749cs.LGcs.AIcs.SE
收藏

LogFormer:用于日志异常检测的预训练与微调流水线

LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection

Beihang University(北京航空航天大学);Cloudwise Research(云智慧研究院);Huawei(华为)

Hongcheng Guo, Jian Yang, Jiaheng Liu, Jiaqi Bai, Boyang Wang, Zhoujun Li, Tieqiao Zheng, Bo Zhang, Junran peng, Qi Tian

AI总结:

针对日志异常检测跨域泛化差的问题,提出基于Transformer的两阶段框架LogFormer,通过预训练共享语义知识、适配器微调迁移参数及Log-Attention模块,在多个数据集上以更少参数和更低成本证明了有效性。

摘要:

日志异常检测是人工智能 IT 运维(AIOps)领域的关键组件。考虑到不同领域的日志数据,在真实工业场景中为未知领域重新训练整个网络是低效的。然而,以往的深度模型仅关注提取同一领域内日志序列的语义,导致在多领域日志上泛化能力差。为缓解此问题,我们提出一个统一的基于 Transformer 的日志异常检测框架(LogFormer),以提升跨不同领域的泛化能力,其中我们建立了一个包含预训练和基于适配器微调阶段的两阶段流程。具体而言,我们的模型首先在源域上预训练,以获取日志数据的共享语义知识。然后,我们通过共享参数将此类知识迁移到目标域。此外,提出了 Log-Attention 模块,以补充日志解析所忽略的信息。所提方法在三个公开数据集和一个真实世界数据集上进行了评估。在多个基准上的实验结果证明了我们 LogFormer 的有效性,且具有更少的可训练参数和更低的训练成本。

英文摘要:

Log anomaly detection is a key component in the field of artificial intelligence for IT operations (AIOps). Considering log data of variant domains, retraining the whole network for unknown domains is inefficient in real industrial scenarios. However, previous deep models merely focused on extracting the semantics of log sequences in the same domain, leading to poor generalization on multi-domain logs. To alleviate this issue, we propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains, where we establish a two-stage process including the pre-training and adapter-based tuning stage. Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer such knowledge to the target domain via shared parameters. Besides, the Log-Attention module is proposed to supplement the information ignored by the log-paring. The proposed method is evaluated on three public and one real-world datasets. Experimental results on multiple benchmarks demonstrate the effectiveness of our LogFormer with fewer trainable parameters and lower training costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑