面向对话摘要的细粒度事实错误标注与检测
Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarization
AI总结：
本文提出首个细粒度对话摘要事实错误标注数据集DIASUMFACT，将错误检测建模为句子级多标签分类，并提出无监督模型ENDERANKER，在性能与SOTA相当的同时资源需求更低。
摘要：
目前已有一系列数据集和模型被提出用于新闻文章等格式良好文档的摘要生成，但对话摘要的研究仍相对不足。本文提出了首个带有细粒度事实错误标注的数据集DIASUMFACT。我们将细粒度事实错误检测定义为一个句子级多标签分类问题，并在该数据集上评估了两个最先进的（SOTA）模型。两个模型的结果均不理想，在6个错误类别上的宏平均F1分数约为0.25。我们进一步提出了一个基于预训练编码器-解码器模型进行候选排序的无监督模型ENDERANKER。该模型在性能上与SOTA模型相当，同时所需资源更少。这些观察结果证实了从对话摘要中检测事实错误所面临的挑战，需要进一步研究，而我们的数据集和结果为后续研究提供了坚实的基础。
英文摘要：
A series of datasets and models have been proposed for summaries generated for well-formatted documents such as news articles. Dialogue summaries, however, have been under explored. In this paper, we present the first dataset with fine-grained factual error annotations named DIASUMFACT. We define fine-grained factual error detection as a sentence-level multi-label classification problem, and we evaluate two state-of-the-art (SOTA) models on our dataset. Both models yield sub-optimal results, with a macro-averaged F1 score of around 0.25 over 6 error classes. We further propose an unsupervised model ENDERANKER via candidate ranking using pretrained encoder-decoder models. Our model performs on par with the SOTA models while requiring fewer resources. These observations confirm the challenges in detecting factual errors from dialogue summaries, which call for further studies, for which our dataset and results offer a solid foundation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
