OpeNLGauge：一种基于开源权重大语言模型的可解释自然语言生成评估指标
OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs
AI总结：
针对现有基于LLM的NLG评估指标依赖专有模型、缺乏细粒度解释的问题，本文提出完全开源的无参考指标OpeNLGauge，其以两阶段开源LLM集成或小型微调模型形式存在，泛化性强，与人类判断相关性具竞争力且解释准确率超两倍。
摘要：
大语言模型（LLMs）作为自然语言生成（NLG）系统的评估器展现出巨大潜力，可实现高质量、无参考且多维度的评估。然而，现有基于LLM的指标存在两大缺陷：依赖专有模型生成训练数据或执行评估，且缺乏细粒度的解释性反馈。本文提出OpeNLGauge，一种完全开源、无参考的NLG评估指标，能基于错误跨度提供准确解释。OpeNLGauge以两阶段更大开源权重LLM集成形式存在，或作为小型微调评估模型，已证实对未见过的任务、领域和维度具有泛化性。广泛的元评估显示，OpeNLGauge与人类判断的相关性具有竞争力，在某些任务上优于最先进模型，同时保持完全可复现性，并提供准确率超两倍的解释。
英文摘要：
Large Language Models (LLMs) have demonstrated great potential as evaluators of NLG systems, allowing for high-quality, reference-free, and multi-aspect assessments. However, existing LLM-based metrics suffer from two major drawbacks: reliance on proprietary models to generate training data or perform evaluations, and a lack of fine-grained, explanatory feedback. In this paper, we introduce OpeNLGauge, a fully open-source, reference-free NLG evaluation metric that provides accurate explanations based on error spans. OpeNLGauge is available as a two-stage ensemble of larger open-weight LLMs, or as a small fine-tuned evaluation model, with confirmed generalizability to unseen tasks, domains and aspects. Our extensive meta-evaluation shows that OpeNLGauge achieves competitive correlation with human judgments, outperforming state-of-the-art models on certain tasks while maintaining full reproducibility and providing explanations more than twice as accurate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
