QuestEval：总结要求基于事实的评估
QuestEval: Summarization Asks for Fact-based Evaluation
摘要：
摘要评价仍然是一个开放的研究问题：目前的指标，如ROUGE是有限的，并与人类的判断相关性差。为了缓解这个问题，最近的工作提出了评估指标，依赖于问答模型来评估摘要是否包含其源文档中的所有相关信息。虽然很有前途，但到目前为止，所提出的方法未能比ROUGE更好地与人类判断相关联。在本文中，我们扩展了以前的方法，并提出了一个统一的框架，名为QuestEval。与ROUGE或BERTScore等已建立的指标相比，QuestEval不需要任何地面实况参考。尽管如此，QuestEval在四个评估维度（一致性，连贯性，流畅性和相关性）上大大提高了与人类判断的相关性，正如我们报告的大量实验所示。
英文摘要：
Summarization evaluation remains an open research problem: current metrics such as ROUGE are known to be limited and to correlate poorly with human judgments. To alleviate this issue, recent work has proposed evaluation metrics which rely on question answering models to assess whether a summary contains all the relevant information in its source document. Though promising, the proposed approaches have so far failed to correlate better than ROUGE with human judgments. In this paper, we extend previous approaches and propose a unified framework, named QuestEval. In contrast to established metrics such as ROUGE or BERTScore, QuestEval does not require any ground-truth reference. Nonetheless, QuestEval substantially improves the correlation with human judgments over four evaluation dimensions (consistency, coherence, fluency, and relevance), as shown in the extensive experiments we report.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
