KELP:通过进化分组树实现鲁棒的在线日志解析
KELP: Robust Online Log Parsing Through Evolutionary Grouping Trees
AI总结:
KELP通过进化分组树实现鲁棒的在线日志解析,克服传统静态模板模型的局限,提升解析准确性和吞吐量。
摘要:
实时日志分析是现代基础设施可观测性的基石。然而,现有的在线解析器在架构上无法适应生产环境的动态性。基于根本静态的模板模型构建,它们极易崩溃:微小的模式漂移会悄然破坏解析流程,导致警报丢失和运维负担。我们提出了KELP(Kelp Evolutionary Log Parser),一种高吞吐量的解析器,其基于一种新的数据结构:进化分组树。与依赖固定规则的启发式方法不同,KELP将模板发现视为一个连续的在线聚类过程。随着日志的到来,树结构会演变,节点会分裂、合并,并根据变化的频率分布重新评估根节点。验证这种适应性需要一个能模拟真实生产复杂性的数据集,但我们将发现标准基准依赖于静态的正则表达式地面真相,无法反映这一点。为了实现严格评估,我们引入了一个新的基准,以反映现代生产系统的结构模糊性。我们的评估表明,KELP在严格的数据集上保持了高精度,而传统启发式方法在此失败,同时不牺牲吞吐量。我们的代码和数据集可在codeberg.org/stonebucklabs/kelp找到。
英文摘要:
Real-time log analysis is the cornerstone of observability for modern infrastructure. However, existing online parsers are architecturally unsuited for the dynamism of production environments. Built on fundamentally static template models, they are dangerously brittle: minor schema drifts silently break parsing pipelines, leading to lost alerts and operational toil. We propose \textbf{KELP} (\textbf{K}elp \textbf{E}volutionary \textbf{L}og \textbf{P}arser), a high-throughput parser built on a novel data structure: the Evolutionary Grouping Tree. Unlike heuristic approaches that rely on fixed rules, KELP treats template discovery as a continuous online clustering process. As logs arrive, the tree structure evolves, nodes split, merge, and re-evaluate roots based on changing frequency distributions. Validating this adaptability requires a dataset that models realistic production complexity, yet we identify that standard benchmarks rely on static, regex-based ground truths that fail to reflect this. To enable rigorous evaluation, we introduce a new benchmark designed to reflect the structural ambiguity of modern production systems. Our evaluation demonstrates that KELP maintains high accuracy on this rigorous dataset where traditional heuristic methods fail, without compromising throughput. Our code and dataset can be found at codeberg.org/stonebucklabs/kelp
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。