MicLog：通过渐进元内省学习实现准确高效的基于大语言模型的日志解析
MicLog: Towards Accurate and Efficient LLM-based Log Parsing via Progressive Meta In-Context Learning
AI总结：
MicLog通过渐进元内省学习提升日志解析准确性和效率，实现更高精度与更快处理速度。
摘要：
日志解析将半结构化日志转换为结构化模板，形成下游分析的关键基础。传统的语法和语义基于解析器在演变日志和数据稀缺性方面面临语义变化和有限领域覆盖的挑战。最近的基于大语言模型（LLM）的解析器利用上下文学习（ICL）从示例中提取语义，显示出更高的准确性。然而，基于LLM的解析器面临两个主要挑战：1）未充分利用ICL能力，特别是动态示例选择和跨领域泛化，导致性能不一致；2）LLM查询耗时且成本高。为了解决这些挑战，我们提出了MicLog，这是首个渐进元内省学习（ProgMeta-ICL）日志解析框架，结合元学习与ICL在小型开源LLM（即Qwen-2.5-3B）上。具体而言，MicLog：i）通过零样本到k样本ProgMeta-ICL范式增强LLM的ICL能力，采用加权DBSCAN候选采样和增强BM25演示选择；ii）通过多级预查询缓存加速解析，动态匹配和优化最近解析的模板。在Loghub-2.0上评估，MicLog比最先进的解析器提高了10.3%的解析准确率，同时将解析时间减少了42.4%。
英文摘要：
Log parsing converts semi-structured logs into structured templates, forming a critical foundation for downstream analysis. Traditional syntax and semantic-based parsers often struggle with semantic variations in evolving logs and data scarcity stemming from their limited domain coverage. Recent large language model (LLM)-based parsers leverage in-context learning (ICL) to extract semantics from examples, demonstrating superior accuracy. However, LLM-based parsers face two main challenges: 1) underutilization of ICL capabilities, particularly in dynamic example selection and cross-domain generalization, leading to inconsistent performance; 2) time-consuming and costly LLM querying. To address these challenges, we present MicLog, the first progressive meta in-context learning (ProgMeta-ICL) log parsing framework that combines meta-learning with ICL on small open-source LLMs (i.e., Qwen-2.5-3B). Specifically, MicLog: i) enhances LLMs' ICL capability through a zero-shot to k-shot ProgMeta-ICL paradigm, employing weighted DBSCAN candidate sampling and enhanced BM25 demonstration selection; ii) accelerates parsing via a multi-level pre-query cache that dynamically matches and refines recently parsed templates. Evaluated on Loghub-2.0, MicLog achieves 10.3% higher parsing accuracy than the state-of-the-art parser while reducing parsing time by 42.4%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
