AmbiFC：基于证据的模糊声明事实核查
AmbiFC: Fact-Checking Ambiguous Claims with Evidence
AI总结：
针对现有事实核查数据集与模型难以处理证据模糊性的问题，研究者构建了含1万条声明、5万段细粒度标注证据的AmbiFC数据集，分析了模糊性相关语言现象，并提出基于软标签的流水线模型，在模糊场景下取得更优性能。
摘要：
自动化事实核查系统通过对照证据验证声明，以预测其真实性。在现实场景中，检索到的证据可能无法明确支持或反驳声明，还会产生相互矛盾但均成立的解读。现有事实核查数据集假定基于其开发的模型会为每条声明预测单一真实性标签，因此不利于处理这类模糊性。为解决该问题，我们提出AmbiFC——一个包含1万条源自现实信息需求的声明的事实核查数据集，其中包含来自5000个维基百科页面的5万段文本的细粒度证据标注。我们分析了AmbiFC中声明与证据比对时由模糊性引发的标注分歧，观察到标注者分歧与指称不足、概率推理等语言现象强相关。我们开发了通过软标签处理这类模糊性的真实性预测模型，发现一个为句子级证据选择和真实性预测学习标签分布的流水线模型取得了最佳性能。我们对比了在AmbiFC不同子集上训练的模型，结果表明，在模糊样本上训练的模型在面对已识别的语言现象时表现更优。
英文摘要：
Automated fact-checking systems verify claims against evidence to predict their veracity. In real-world scenarios, the retrieved evidence may not unambiguously support or refute the claim and yield conflicting but valid interpretations. Existing fact-checking datasets assume that the models developed with them predict a single veracity label for each claim, thus discouraging the handling of such ambiguity. To address this issue we present AmbiFC, a fact-checking dataset with 10k claims derived from real-world information needs. It contains fine-grained evidence annotations of 50k passages from 5k Wikipedia pages. We analyze the disagreements arising from ambiguity when comparing claims against evidence in AmbiFC, observing a strong correlation of annotator disagreement with linguistic phenomena such as underspecification and probabilistic reasoning. We develop models for predicting veracity handling this ambiguity via soft labels and find that a pipeline that learns the label distribution for sentence-level evidence selection and veracity prediction yields the best performance. We compare models trained on different subsets of AmbiFC and show that models trained on the ambiguous instances perform better when faced with the identified linguistic phenomena.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
