CsFEVER 与 CTKFacts：获取用于事实验证的捷克语数据
CsFEVER and CTKFacts: Acquiring Czech data for fact verification
AI总结：
本文构建并发布捷克语事实验证数据集 CsFEVER 与 CTKFacts，结合翻译对齐和人工标注，分析标注偏差并提供基线模型与工具。
摘要：
在本文中，我们考察了为自动化事实核查获取捷克语数据的若干方法；该任务通常被建模为依据可信真值语料库对文本声明真实性进行分类。我们尝试收集由事实性声明、真值语料库中的证据及其真实性标签（支持、反驳或信息不足）组成的数据集。作为首次尝试，我们基于 Wikipedia 语料库生成了大规模 FEVER 数据集的捷克语版本。我们采用机器翻译与文档对齐相结合的混合方法；所提出的方法及提供的工具可轻松应用于其他语言。我们讨论了其弱点与不准确之处，提出了未来用于清理这些问题的方法，并发布了由此得到的 12.7 万条翻译结果，以及一个可可靠用于自然语言推理任务的数据集版本——CsFEVER-NLI。此外，我们收集了一个包含 3,097 条声明的新颖数据集，并使用捷克通讯社的 220 万篇文章语料库进行标注。我们提出了基于 FEVER 方法的扩展标注方法；由于底层语料库属于商业秘密，我们还发布了一个可独立用于自然语言推理任务的数据集版本，称为 CTKFactsNLI。我们分析了两个已获取数据集中的虚假线索，即会导致模型过拟合的标注模式。CTKFacts 还接受了标注者间一致性检查，经过彻底清理，并提炼出常见标注者错误的类型学。最后，我们为事实核查流程的所有阶段提供了基线模型，并发布了 NLI 数据集、标注平台及其他实验数据。
英文摘要：
In this paper, we examine several methods of acquiring Czech data for automated fact-checking, which is a task commonly modeled as a classification of textual claim veracity w.r.t. a corpus of trusted ground truths. We attempt to collect sets of data in form of a factual claim, evidence within the ground truth corpus, and its veracity label (supported, refuted or not enough info). As a first attempt, we generate a Czech version of the large-scale FEVER dataset built on top of Wikipedia corpus. We take a hybrid approach of machine translation and document alignment; the approach and the tools we provide can be easily applied to other languages. We discuss its weaknesses and inaccuracies, propose a future approach for their cleaning and publish the 127k resulting translations, as well as a version of such dataset reliably applicable for the Natural Language Inference task - the CsFEVER-NLI. Furthermore, we collect a novel dataset of 3,097 claims, which is annotated using the corpus of 2.2M articles of Czech News Agency. We present its extended annotation methodology based on the FEVER approach, and, as the underlying corpus is kept a trade secret, we also publish a standalone version of the dataset for the task of Natural Language Inference we call CTKFactsNLI. We analyze both acquired datasets for spurious cues - annotation patterns leading to model overfitting. CTKFacts is further examined for inter-annotator agreement, thoroughly cleaned, and a typology of common annotator errors is extracted. Finally, we provide baseline models for all stages of the fact-checking pipeline and publish the NLI datasets, as well as our annotation platform and other experimental data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
