REFLEX：基于大语言模型判断的无参考日志摘要评估
REFLEX: Reference-Free Evaluation of Log Summarization via Large Language Model Judgment
AI总结：
REFLEX通过大语言模型评估日志摘要质量，无需参考摘要或人工标注，提供稳定且细粒度的评估结果，优于传统指标。
摘要：
评估日志摘要系统具有挑战性，因为缺乏高质量参考摘要，且现有指标如ROUGE和BLEU依赖表面词汇重叠。我们引入REFLEX，一种基于大语言模型（LLM）判断的无参考日志摘要评估指标。REFLEX利用LLM作为零样本评估者，沿相关性、信息量和连贯性等维度评估摘要质量，无需黄金标准参考或人工标注。我们证明REFLEX在多个日志摘要数据集上产生稳定、可解释且细粒度的评估结果，并比传统指标更有效地区分模型输出。REFLEX为在参考数据稀缺或不可用的现实场景中评估日志摘要提供了可扩展的替代方案。
英文摘要：
Evaluating log summarization systems is challenging due to the lack of high-quality reference summaries and the limitations of existing metrics like ROUGE and BLEU, which depend on surface-level lexical overlap. We introduce REFLEX, a reference-free evaluation metric for log summarization based on large language model (LLM) judgment. REFLEX uses LLMs as zero-shot evaluators to assess summary quality along dimensions such as relevance, informativeness, and coherence, without requiring gold-standard references or human annotations. We show that REFLEX produces stable, interpretable, and fine-grained evaluations across multiple log summarization dataset, and more effectively distinguishes model outputs than traditional metrics. REFLEX provides a scalable alternative for evaluating log summaries in real-world settings where reference data is scarce or unavailable.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
