FactKB：使用以事实知识增强的语言模型进行可泛化的事实性评估
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge
AI总结：
本文提出FactKB，一种基于外部知识库事实预训练语言模型的事实性评估方法，通过三种互补预训练目标提升跨领域泛化能力，在多个数据集上达到SOTA，有效检测实体与关系错误。
摘要：
评估自动生成摘要的事实一致性对于可靠摘要系统的发展与采用至关重要。尽管近期取得了一些进展，但现有的事实性评估模型并不稳健，尤其容易在新领域中产生实体和关系错误。我们提出了FactKB，这是一种简单且新的事实性评估方法，能够跨领域泛化，特别是在实体和关系方面。FactKB基于使用从外部知识库中提取的事实进行预训练的语言模型。我们引入了三种互补的事实性预训练目标，分别基于直接实体事实、基于实体辅助知识的事实，以及通过知识库游走组合构建的事实。由此产生的事实性评估模型在两个领域内新闻摘要基准和三个领域外科学文献数据集上均实现了最先进的性能。对FactKB的进一步分析表明，该模型在检测摘要中错误实体和关系方面的能力有所提升，并且在跨领域时表现出稳健性和泛化能力。
英文摘要：
Evaluating the factual consistency of automatically generated summaries is essential for the progress and adoption of reliable summarization systems. Despite recent advances, existing factuality evaluation models are not robust, being especially prone to entity and relation errors in new domains. We propose FactKB, a simple new approach to factuality evaluation that is generalizable across domains, in particular with respect to entities and relations. FactKB is based on language models pretrained using facts extracted from external knowledge bases. We introduce three types of complementary factuality pretraining objectives based on direct entity facts, facts grounded in auxiliary knowledge about entities, and facts constructed compositionally through knowledge base walks. The resulting factuality evaluation model achieves state-of-the-art performance on two in-domain news summarization benchmarks as well as on three out-of-domain scientific literature datasets. Further analysis of FactKB shows improved ability to detect erroneous entities and relations in summaries and is robust and generalizable across domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
