Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models
Accenture（埃森哲）；University of Maryland（马里兰大学）
Evan Williams, Paul Rodrigues, Valerie Novak
备注：
To Appear As: Evan Williams, Paul Rodrigues, Valerie Novak. Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models. In: Cappellato et al. Working Notes of CLEF 2020-Conference and Labs of the Evaluation Forum. Thessaloniki, Greece. 22-25 September 2020
We introduce the strategies used by the Accenture Team for the CLEF2020 CheckThat! Lab, Task 1, on English and Arabic. This shared task evaluated whether a claim in social media text should be professionally fact checked. To a journalist, a statement presented as fact, which would be of interest to a large audience, requires professional fact-checking before dissemination. We utilized BERT and RoBERTa models to identify claims in social media text a professional fact-checker should review, and rank these in priority order for the fact-checker. For the English challenge, we fine-tuned a RoBERTa model and added an extra mean pooling layer and a dropout layer to enhance generalizability to unseen text. For the Arabic task, we fine-tuned Arabic-language BERT models and demonstrate the use of back-translation to amplify the minority class and balance the dataset. The work presented here was scored 1st place in the English track, and 1st, 2nd, 3rd, and 4th place in the Arabic track.
This paper discusses the approach used by the Accenture Team for CLEF2021 CheckThat! Lab, Task 1, to identify whether a claim made in social media would be interesting to a wide audience and should be fact-checked. Twitter training and test data were provided in English, Arabic, Spanish, Turkish, and Bulgarian. Claims were to be classified (check-worthy/not check-worthy) and ranked in priority order for the fact-checker. Our method used deep neural network transformer models with contextually sensitive lexical augmentation applied on the supplied training datasets to create additional training…
The CLEF-2026 CheckThat! Lab: Advancing Multilingual Fact-Checking
University of Applied Sciences Potsdam, Germany（应用科学大学波茨坦，德国）；GESIS - Leibniz Institute for the Social Sciences, Cologne, Germany（GESIS—社会科学院莱比锡研究所，德国）；Stockholm University, Sweden（斯德哥尔摩大学，瑞典）；University of Stavanger, Norway（斯塔万格大学，挪威）；Indian Institute of Technology Delhi, India（印度理工学院德里，印度）；Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi（穆罕默德·本·扎耶德人工智能大学，阿布扎比）；Delft University of Technology, The Netherlands（代尔夫特理工大学，荷兰）；Independent Researcher（独立研究者）；médialab Sciences Po, Paris, France（Sciences Po 媒介实验室，法国）；University of Montpellier, LIRMM, CNRS, Montpellier, France（蒙彼利埃大学，LIRMM，CNRS，法国）
The CLEF-2025 CheckThat! Lab: Subjectivity, Fact-Checking, Claim Normalization, and Retrieval
Qatar Computing Research Institute（卡塔尔计算研究所）；University of Applied Sciences Potsdam（波茨坦应用科学大学）；University of Bologna（博洛尼亚大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Indian Institute of Technology Delhi（印度理工学院德里分校）；GESIS - Leibniz Institute for the Social Sciences（GESIS莱布尼茨社会科学研究所）；Heinrich-Heine-University Düsseldorf（杜塞尔多夫海因里希·海涅大学）；Delft University of Technology（代尔夫特理工大学）；Indraprastha Institute of Information Technology（因德拉普拉斯塔信息技术学院）；University of Montpellier（蒙彼利埃大学）
SiteGround；Amazon（亚马逊）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）；Qatar University（卡塔尔大学）；College of Humanities and Social Sciences, HBKU（哈马德·本·哈利法大学人文与社会科学学院）；Sofia University “St Kliment Ohridski”（索非亚大学“圣克里门特·奥赫里德斯基”）
We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 1: Check-Worthiness. The task asks to predict which claims in a political debate should be prioritized for fact-checking. In particular, given a debate or a political speech, the goal was to produce a ranked list of its sentences based on their worthiness for fact checking. We offered the task in both English and Arabic, based on debates from the 2016 US Presidential Campaign, as well as on some speeches during and after the campaign. A total of 30 teams r…
Fraunhofer SIT at CheckThat! 2023: Tackling Classification Uncertainty Using Model Souping on the Example of Check-Worthiness Classification
Fraunhofer Institute for Secure Information Technology SIT（弗劳恩霍夫安全信息技术研究所）；ATHENE - National Research Center for Applied Cybersecurity（ATHENE——国家应用网络安全研究中心）
Fraunhofer SIT 团队在 CLEF-2023 CheckThat! Task 1B 中提出基于 Model Souping 的集成分类方法，用于政治辩论文本的值得核查性分类，在英语数据集上取得 0.878 的 F1 分数，排名第二。
AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization
Institute of Informatics, Federal University of Goiás, Brazil（信息学院，戈亚斯联邦大学，巴西）；Advanced Knowledge Center in Immersive Technology (AKCIT), Federal University of Goiás, Brazil（沉浸式技术高级知识中心（AKCIT），戈亚斯联邦大学，巴西）
Identifying check-worthy claims is often the first step of automated fact-checking systems. Tackling this task in a multilingual setting has been understudied. Encoding inputs with multilingual text representations could be one approach to solve the multilingual check-worthiness detection. However, this approach could suffer if cultural bias exists within the communities on determining what is check-worthy.In this paper, we propose a language identification task as an auxiliary task to mitigate unintended bias.With this purpose, we experiment joint training by using the datasets from CLEF-2021…
Sofia University “St Kliment Ohridski”（索非亚大学“圣克里门特·奥赫里德斯基”）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
While misinformation and disinformation have been thriving in social media for years, with the emergence of the COVID-19 pandemic, the political and the health misinformation merged, thus elevating the problem to a whole new level and giving rise to the first global infodemic. The fight against this infodemic has many aspects, with fact-checking and debunking false and misleading claims being among the most important ones. Unfortunately, manual fact-checking is time-consuming and automatic fact-checking is resource-intense, which means that we need to pre-filter the input social media posts an…
TIB Leibniz Information Centre for Science and Technology（莱布尼茨科技信息中心）；L3S Research Center, Leibniz University Hannover（汉诺威莱布尼茨大学L3S研究中心）
In this digital age of news consumption, a news reader has the ability to react, express and share opinions with others in a highly interactive and fast manner. As a consequence, fake news has made its way into our daily life because of very limited capacity to verify news on the Internet by large companies as well as individuals. In this paper, we focus on solving two problems which are part of the fact-checking ecosystem that can help to automate fact-checking of claims in an ever increasing stream of content on social media. For the first problem, claim check-worthiness prediction, we explo…
Computer Science and Engineering Department, Qatar University（卡塔尔大学计算机科学与工程系）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）；DIT, Università di Bologna（博洛尼亚大学DIT（此处DIT按原文保留，若需全称需更多上下文））；Department of Computer Science, University of Copenhagen（哥本哈根大学计算机科学系）
We present an overview of the second edition of the CheckThat! Lab at CLEF 2019. The lab featured two tasks in two different languages: English and Arabic. Task 1 (English) challenged the participating systems to predict which claims in a political debate or speech should be prioritized for fact-checking. Task 2 (Arabic) asked to (A) rank a given set of Web pages with respect to a check-worthy claim based on their usefulness for fact-checking that claim, (B) classify these same Web pages according to their degree of usefulness for fact-checking the target claim, (C) identify useful passages fr…
DIT–Università di Bologna（博洛尼亚大学数字与信息工程系）；Qatar University（卡塔尔大学）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
We describe the third edition of the CheckThat! Lab, which is part of the 2020 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes four complementary tasks and a related task from previous lab editions, offered in English, Arabic, and Spanish. Task 1 asks to predict which tweets in a Twitter stream are worth fact-checking. Task 2 asks to determine whether a claim posted in a tweet can be verified using a set of previously fact-checked claims. Task 3 asks to retrieve text snippets from a given set of Web pages that would be useful for verifying a target tweet's claim. Task 4 asks to pre…
Daffodil International University（达福迪尔国际大学）；Qatar Computing Research Institute（卡塔尔计算研究所）
The wide use of social media and digital technologies facilitates sharing various news and information about events and activities. Despite sharing positive information misleading and false information is also spreading on social media. There have been efforts in identifying such misleading information both manually by human experts and automatic tools. Manual effort does not scale well due to the high volume of information, containing factual claims, are appearing online. Therefore, automatically identifying check-worthy claims can be very useful for human experts. In this study, we describe …
Fraunhofer SIT at CheckThat! 2023: Mixing Single-Modal Classifiers to Estimate the Check-Worthiness of Multi-Modal Tweets
Fraunhofer Institute for Secure Information Technology SIT（弗劳恩霍夫安全信息技术研究所）；ATHENE - National Research Center for Applied Cybersecurity（ATHENE——国家应用网络安全研究中心）
Qatar Computing Research Institute HBKU（卡塔尔计算研究所（HBKU））；University of Padova（帕多瓦大学）；Qatar University（卡塔尔大学）；DIT Università di Bologna（博洛尼亚大学DIT学院）；Newtral Media Audiovisual；Sofia University（索非亚大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）；University of Applied Sciences Potsdam（波茨坦应用科学大学）；University of Hildesheim（希尔德斯海姆大学）；TOBB University of Economics and Technology（托普卡帕经济与技术大学）
We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims (in Arabic and English). Task 3 asks to predict the veracity of a news article and its topical…
Sofia University “St. Kliment Ohridski”（圣基里尔·奥赫里德斯基索菲亚大学）；HSE University（高等经济大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
While there has been substantial progress in developing systems to automate fact-checking, they still lack credibility in the eyes of the users. Thus, an interesting approach has emerged: to perform automatic fact-checking by verifying whether an input claim has been previously fact-checked by professional fact-checkers and to return back an article that explains their decision. This is a sensible approach as people trust manual fact-checking, and as many claims are repeated multiple times. Yet, a major issue when building such systems is the small number of known tweet--verifying article pair…
