FaVIQ：从信息寻求问题中进行事实验证
FaVIQ: FAct Verification from Information-seeking Questions
摘要：
尽管人们对开发通用事实检查模型有很大的兴趣，但要构建一个具有现实真实世界声明的大规模事实验证数据集仍然具有挑战性。现有的声明要么是由众包工作者撰写的，从而引入了难以控制的微妙偏见，要么是由专业事实核查人员手动验证，导致它们昂贵且规模有限。在本文中，我们构建了一个名为FAVIQ的大规模具有挑战性的事实验证数据集，该数据集由来自现有的模糊信息搜索问题语料库的188k索赔组成。问题中的模糊性使得能够自动构建反映用户困惑的真实和虚假声明（例如，电影拍摄的年份与上映的年份）。FAVIQ中的声明被证实是自然的，包含很少的词汇偏见，并需要对证据的全面理解进行验证。我们的实验表明，最先进的模型远远不能解决我们的新任务。此外，对我们的数据进行训练有助于专业的事实检查，比在广泛使用的数据集FEVER或域内数据上训练的模型的绝对性能高出17%。总而言之，我们的数据将成为自然语言理解的一个具有挑战性的基准，并支持专业事实核查的未来进展。
英文摘要：
Despite significant interest in developing general purpose fact checking models, it is challenging to construct a large-scale fact verification dataset with realistic real-world claims. Existing claims are either authored by crowdworkers, thereby introducing subtle biases that are difficult to control for, or manually verified by professional fact checkers, causing them to be expensive and limited in scale. In this paper, we construct a large-scale challenging fact verification dataset called FAVIQ, consisting of 188k claims derived from an existing corpus of ambiguous information-seeking questions. The ambiguities in the questions enable automatically constructing true and false claims that reflect user confusions (e.g., the year of the movie being filmed vs. being released). Claims in FAVIQ are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification. Our experiments show that the state-of-the-art models are far from solving our new task. Moreover, training on our data helps in professional fact-checking, outperforming models trained on the widely used dataset FEVER or in-domain data by up to 17% absolute. Altogether, our data will serve as a challenging benchmark for natural language understanding and support future progress in professional fact checking.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
