Identification of appropriate supporting evidence is critical to the success of scientific fact checking. However, existing approaches rely on off-the-shelf Information Retrieval algorithms that rank documents based on relevance rather than the evidence they provide to support or refute the claim being checked. This paper proposes +VeriRel which includes verification success in the document ranking. Experimental results on three scientific fact checking datasets (SciFact, SciFact-Open and Check-Covid) demonstrate consistently leading performance by +VeriRel for document evidence retrieval and a positive impact on downstream verification. This study highlights the potential of integrating verification feedback to document relevance assessment for effective scientific fact checking systems. It shows promising future work to evaluate fine-grained relevance when examining complex documents for advanced scientific fact checking.
MIT Computer Science and Artificial Intelligence Laboratory（麻省理工学院计算机科学与人工智能实验室）
We present FAKTA which is a unified framework that integrates various components of a fact checking process: document retrieval from media sources with various types of reliability, stance detection of documents with respect to given claims, evidence extraction, and linguistic analysis. FAKTA predicts the factuality of given claims and provides evidence at the document and sentence level to explain its predictions
AfrIFact: Cultural Information Retrieval, Evidence Extraction and Fact Checking for African Languages
Masakhane NLP；Saarland University（萨尔兰大学）；Saarland Informatic Campus（萨尔兰信息学院）；National Institute of Informatics（国家信息研究所）；University of Waterloo（滑铁卢大学）；University College London（伦敦大学学院）；Instituto Politécnico Nacional（国家理工学院）；University of Ilorin（伊洛林大学）；Universidad de Deusto（德尤斯大学）；Bayero University（贝亚罗大学）；University of Lagos（拉各斯大学）；Black Swan（黑天鹅）；Addis Ababa University（亚的斯亚贝巴大学）；Universite Alioune Diop（阿利乌内·迪普大学）；McGill University（麦吉尔大学）
Automated fact-checking based on machine learning is a promising approach to identify false information distributed on the web. In order to achieve satisfactory performance, machine learning methods require a large corpus with reliable annotations for the different tasks in the fact-checking process. Having analyzed existing fact-checking corpora, we found that none of them meets these criteria in full. They are either too small in size, do not provide detailed annotations, or are limited to a single domain. Motivated by this gap, we present a new substantially sized mixed-domain corpus with a…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Institute of Information Science, Academia Sinica（中央研究院信息科学研究所）；National Tsing Hua University（国立清华大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；Institute of Information, Academia Sinica（中央研究院资讯研究所）
Automated fact checking systems have been proposed that quickly provide veracity prediction at scale to mitigate the negative influence of fake news on people and on public opinion. However, most studies focus on veracity classifiers of those systems, which merely predict the truthfulness of news articles. We posit that effective fact checking also relies on people's understanding of the predictions. In this paper, we propose elucidating fact checking predictions using counterfactual explanations to help people understand why a specific piece of news was identified as fake. In this work, gener…
Improving Reliability and Explainability of Medical Question Answering through Atomic Fact Checking in Retrieval-Augmented LLMs
Technical University of Munich（慕尼黑工业大学）；TUM University Hospital Rechts der Isar（慕尼黑工业大学 Rechts der Isar 大学医院）；German Heart Center Munich（慕尼黑德国心脏中心）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）；German Consortium for Translational Cancer Research（德国转化癌症研究联盟）
Priberam Labs（Priberam实验室）；Priberam Informática, S.A.（Priberam信息公司）；Dep. of Computer Science and Technology, University of Cambridge（剑桥大学计算机科学与技术系）；BBC News Labs（英国广播公司新闻实验室）
Fact checking is an essential task in journalism; its importance has been highlighted due to recently increased concerns and efforts in combating misinformation. In this paper, we present an automated fact-checking platform which given a claim, it retrieves relevant textual evidence from a document collection, predicts whether each piece of evidence supports or refutes the claim, and returns a final verdict. We describe the architecture of the system and the user interface, focusing on the choices made to improve its user-friendliness and transparency. We conduct a user study of the fact-check…
L3S Research Center, Leibniz University of Hannover（莱布尼茨汉诺威大学L3S研究中心）；Institute of Computer Science, FORTH-ICS（希腊研究与技术基金会计算机科学研究所）；Dept. of CSE, IIT Patna（印度理工学院巴特那分校计算机科学与工程系）；Chongqing University of Technology（重庆理工大学）；GESIS - Leibniz Institute for the Social Sciences, Cologne & Heinrich-Heine-University Düsseldorf（科隆莱布尼茨社会科学研究所暨杜塞尔多夫海因里希·海涅大学）
Fact checking is an essential challenge when combating fake news. Identifying documents that agree or disagree with a particular statement (claim) is a core task in this process. In this context, stance detection aims at identifying the position (stance) of a document towards a claim. Most approaches address this task through a 4-class classification model where the class distribution is highly imbalanced. Therefore, they are particularly ineffective in detecting the minority classes (for instance, 'disagree'), even though such instances are crucial for tasks such as fact-checking by providing…
Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）；University of Padova（帕多瓦大学）
Recent years have seen the proliferation of disinformation and fake news online. Traditional approaches to mitigate these issues is to use manual or automatic fact-checking. Recently, another approach has emerged: checking whether the input claim has previously been fact-checked, which can be done automatically, and thus fast, while also offering credibility and explainability, thanks to the human fact-checking and explanations in the associated fact-checking article. Here, we focus on claims made in a political debate and we study the impact of modeling the context of the claim: both on the s…
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
Getting an overview over the legal domain has become challenging, especially in a broad, international context. Legal question answering systems have the potential to alleviate this task by automatically retrieving relevant legal texts for a specific statement and checking whether the meaning of the statement can be inferred from the found documents. We investigate a combination of the BM25 scoring method of Elasticsearch with word embeddings trained on English translations of the German and Japanese civil law. For this, we define criteria which select a dynamic number of relevant documents ac…
Cornell University（康奈尔大学）；Sofia University（索非亚大学）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）；University of Padova（帕多瓦大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Given the recent proliferation of false claims online, there has been a lot of manual fact-checking effort. As this is very time-consuming, human fact-checkers can benefit from tools that can support them and make them more efficient. Here, we focus on building a system that could provide such support. Given an input document, it aims to detect all sentences that contain a claim that can be verified by some previously fact-checked claims (from a given database). The output is a re-ranked list of the document sentences, so that those that can be verified are ranked as high as possible, together…
Sofia University “St. Kliment Ohridski”（索非亚大学“圣克里门特·奥赫里德斯基”）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
In the context of investigative journalism, we address the problem of automatically identifying which claims in a given document are most worthy and should be prioritized for fact-checking. Despite its importance, this is a relatively understudied problem. Thus, we create a new dataset of political debates, containing statements that have been fact-checked by nine reputable sources, and we train machine learning models to predict which claims should be prioritized for fact-checking, i.e., we model the problem as a ranking task. Unlike previous work, which has looked primarily at sentences in i…
