SEval-Ex：面向可解释摘要评估的语句级框架
SEval-Ex: A Statement-Level Framework for Explainable Summarization Evaluation
AI总结：
SEval-Ex框架通过将摘要评估分解为原子语句并进行语句级匹配，在保持可解释性的同时实现了优于GPT-4评估器的最先进性能。
摘要：
评估文本摘要质量仍然是自然语言处理中的一个关键挑战。当前方法在性能与可解释性之间面临权衡。我们提出SEval-Ex框架，通过将摘要评估分解为原子语句来弥合这一差距，同时实现高性能和可解释性。SEval-Ex采用两阶段流程：首先使用LLM从文本源和摘要中提取原子语句，然后在生成的语句之间进行匹配。与仅提供摘要级分数的现有方法不同，我们的方法通过语句级对齐为其决策生成详细证据。在SummEval基准上的实验表明，SEval-Ex在一致性上与人类一致性判断的相关性达到0.580，取得了最先进的性能，超越了基于GPT-4的评估器（0.521），同时保持了可解释性。最后，我们的框架展示了对幻觉的鲁棒性。
英文摘要：
Evaluating text summarization quality remains a critical challenge in Natural Language Processing. Current approaches face a trade-off between performance and interpretability. We present SEval-Ex, a framework that bridges this gap by decomposing summarization evaluation into atomic statements, enabling both high performance and explainability. SEval-Ex employs a two-stage pipeline: first extracting atomic statements from text source and summary using LLM, then a matching between generated statements. Unlike existing approaches that provide only summary-level scores, our method generates detailed evidence for its decisions through statement-level alignments. Experiments on the SummEval benchmark demonstrate that SEval-Ex achieves state-of-the-art performance with 0.580 correlation on consistency with human consistency judgments, surpassing GPT-4 based evaluators (0.521) while maintaining interpretability. Finally, our framework shows robustness against hallucination.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
