提高标题生成的真实性
Improving Truthfulness of Headline Generation
摘要：
大多数关于抽象摘要的研究报告了系统摘要和参考摘要之间的ROUGE评分。然而，我们有一个问题的真实性生成的摘要：是否所有的事实生成的摘要中提到的源文本。本文在两个流行的数据集上探讨了如何提高标题生成的真实性。分析由最先进的编码器-解码器模型生成的标题，我们发现该模型有时会生成不真实的标题。我们推测原因之一在于用于训练模型的监管数据不真实。为了量化文章标题对的真实性，我们考虑了文章是否包含其标题的文本蕴涵。在确认数据集中存在大量不真实实例后，本研究假设从监督数据中删除不真实实例可以弥补模型的不真实行为问题。建立一个二元分类器，预测文章和标题之间的蕴涵关系，我们过滤掉监督数据中的不真实实例。实验结果表明，基于过滤监督数据训练的标题生成模型在ROUGE评分上没有明显差异，但在自动和人工评价标题方面有显著提高。
英文摘要：
Most studies on abstractive summarization report ROUGE scores between system and reference summaries. However, we have a concern about the truthfulness of generated summaries: whether all facts of a generated summary are mentioned in the source text. This paper explores improving the truthfulness in headline generation on two popular datasets. Analyzing headlines generated by the state-of-the-art encoder-decoder model, we show that the model sometimes generates untruthful headlines. We conjecture that one of the reasons lies in untruthful supervision data used for training the model. In order to quantify the truthfulness of article-headline pairs, we consider the textual entailment of whether an article entails its headline. After confirming quite a few untruthful instances in the datasets, this study hypothesizes that removing untruthful instances from the supervision data may remedy the problem of the untruthful behaviors of the model. Building a binary classifier that predicts an entailment relation between an article and its headline, we filter out untruthful instances from the supervision data. Experimental results demonstrate that the headline generation model trained on filtered supervision data shows no clear difference in ROUGE scores but remarkable improvements in automatic and manual evaluations of the generated headlines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
