arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.01796cs.SE
收藏

LILAC:使用具有自适应解析缓存的LLM进行日志解析

LILAC: Log Parsing using LLMs with Adaptive Parsing Cache

Zhihan Jiang, Jinyang Liu, Zhuangbin Chen, Yichen Li, Junjie Huang, Yintong Huo, Pinjia He, Jiazhen Gu, Michael R. Lyu

AI总结:

提出首个基于自适应解析缓存的LLM日志解析框架LILAC,利用上下文学习和分层候选采样提升准确性,通过缓存机制降低查询开销,在公开数据集上模板准确率F1分数提升69.5%且效率媲美最快基线。

摘要:

日志解析将日志消息转换为结构化格式,是各种日志分析任务的前提步骤。尽管已有多种日志解析方法被提出,但由于使用人工制定的规则或训练数据有限的基于学习的模型,它们在复杂日志数据上的性能仍受影响。近期出现的强大大型语言模型(LLMs)展示了与代码和日志相关的丰富预训练知识,使得将LLMs应用于日志解析具有前景。然而,它们目前缺乏专门的日志解析能力,阻碍了解析的准确性。此外,固有的不一致答案以及巨大的开销,阻碍了基于LLM的日志解析的实际应用。为了应对这些挑战,我们提出了LILAC,这是首个使用具有自适应解析缓存的LLM的实用日志解析框架。为实现准确且鲁棒的日志解析,LILAC通过执行分层候选采样算法和选择高质量演示,利用了LLM的上下文学习(ICL)能力。此外,LILAC引入了一个新组件,即自适应解析缓存,用于存储和优化LLM生成的模板。它通过实现先前处理过的日志模板的快速检索,帮助缓解LLM的低效问题。在此过程中,LILAC自适应地更新解析缓存中的模板,以确保解析结果的一致性。在公开的大规模数据集上的广泛评估表明,LILAC在模板准确率的平均F1分数上比最先进的方法高出69.5%。此外,LILAC将对LLM的查询次数减少了数个数量级,实现了与最快基线相当的效率。

英文摘要:

Log parsing transforms log messages into structured formats, serving as the prerequisite step for various log analysis tasks. Although a variety of log parsing approaches have been proposed, their performance on complicated log data remains compromised due to the use of human-crafted rules or learning-based models with limited training data. The recent emergence of powerful large language models (LLMs) demonstrates their vast pre-trained knowledge related to code and logging, making it promising to apply LLMs for log parsing. However, their lack of specialized log parsing capabilities currently hinders their accuracy in parsing. Moreover, the inherent inconsistent answers, as well as the substantial overhead, prevent the practical adoption of LLM-based log parsing. To address these challenges, we propose LILAC, the first practical log parsing framework using LLMs with adaptive parsing cache. To facilitate accurate and robust log parsing, LILAC leverages the in-context learning (ICL) capability of the LLM by performing a hierarchical candidate sampling algorithm and selecting high-quality demonstrations. Furthermore, LILAC incorporates a novel component, an adaptive parsing cache, to store and refine the templates generated by the LLM. It helps mitigate LLM's inefficiency issue by enabling rapid retrieval of previously processed log templates. In this process, LILAC adaptively updates the templates within the parsing cache to ensure the consistency of parsed results. The extensive evaluation on public large-scale datasets shows that LILAC outperforms state-of-the-art methods by 69.5% in terms of the average F1 score of template accuracy. In addition, LILAC reduces the query times to LLMs by several orders of magnitude, achieving a comparable efficiency to the fastest baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑