LLM-SrcLog: 通过大语言模型实现前瞻性与统一的日志模板提取
LLM-SrcLog: Towards Proactive and Unified Log Template Extraction via Large Language Models
AI总结：
LLM-SrcLog通过结合源代码分析与大语言模型，实现前瞻性日志模板提取，提升解析效率和准确性。
摘要：
日志解析将原始日志转换为包含常量和变量的结构化模板。它支撑着异常检测、故障诊断和其他AIOps任务。当前的解析器大多是反应式且日志中心化的。它们只从日志中推断模板，大多忽略了源代码。这限制了它们捕捉动态日志结构或适应演进系统的潜力。此外，每个日志的LLM推理对于实际部署来说成本太高。在本文中，我们提出了LLM-SrcLog，一种前瞻性且统一的日志模板解析框架。它在部署前直接从源代码中提取模板，并通过数据驱动的解析补充没有可用代码的日志。LLM-SrcLog集成了一个跨功能静态代码分析器以重建有意义的日志上下文，一个基于LLM的白盒模板提取器，带有后处理步骤以区分常量和变量，以及一个黑盒模板提取器，该提取器结合了数据驱动的聚类方法来处理剩余未匹配的日志。在两个公开基准（Hadoop和Zookeeper）和一个大规模工业系统（Sunfire-Compute）上的实验表明，与两个基于LLM的基线相比，LLM-SrcLog在平均F1分数上提高了2-17%和8-35%。同时，其在线解析延迟与数据驱动方法相当，比每个日志的LLM解析快约1000倍。LLM-SrcLog在速度和准确性之间实现了接近理想的平衡。最后，我们通过实际案例研究在真实生产环境中进一步验证了LLM-SrcLog的有效性。
英文摘要：
Log parsing transforms raw logs into structured templates containing constants and variables. It underpins anomaly detection, failure diagnosis, and other AIOps tasks. Current parsers are mostly reactive and log-centric. They only infer templates from logs, mostly overlooking the source code. This restricts their capacity to grasp dynamic log structures or adjust to evolving systems. Moreover, per-log LLM inference is too costly for practical deployment. In this paper, we propose LLM-SrcLog, a proactive and unified framework for log template parsing. It extracts templates directly from source code prior to deployment and supplements them with data-driven parsing for logs without available code. LLM-SrcLog integrates a cross-function static code analyzer to reconstruct meaningful logging contexts, an LLM-based white-box template extractor with post-processing to distinguish constants from variables, and a black-box template extractor that incorporates data-driven clustering for remaining unmatched logs. Experiments on two public benchmarks (Hadoop and Zookeeper) and a large-scale industrial system (Sunfire-Compute) show that, compared to two LLM-based baselines, LLM-SrcLog improves average F1-score by 2-17% and 8-35%. Meanwhile, its online parsing latency is comparable to data-driven methods and about 1,000 times faster than per-log LLM parsing. LLM-SrcLog achieves a near-ideal balance between speed and accuracy. Finally, we further validate the effectiveness of LLM-SrcLog through practical case studies in a real-world production environment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
