首页/论文/2004.08166

arXiv 2004.081662020-04cs.CL

Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness

TOBB University of Economics and Technology（TOBB经济与技术大学）

Yavuz Selim Kartal, Busra Guvenen, Mucahid Kutlu

英文摘要：

The massive amount of misinformation spreading on the Internet on a daily basis has enormous negative impacts on societies. Therefore, we need automated systems helping fact-checkers in the combat against misinformation. In this paper, we propose a model prioritizing the claims based on their check-worthiness. We use BERT model with additional features including domain-specific controversial topics, word embeddings, and others. In our experiments, we show that our proposed model outperforms all state-of-the-art models in both test collections of CLEF Check That! Lab in 2018 and 2019. We also conduct a qualitative analysis to shed light-detecting check-worthy claims. We suggest requesting rationales behind judgments are needed to understand subjective nature of the task and problematic labels.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness

SiteGround；Amazon（亚马逊）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）；Qatar University（卡塔尔大学）；College of Humanities and Social Sciences, HBKU（哈马德·本·哈利法大学人文与社会科学学院）；Sofia University “St Kliment Ohridski”（索非亚大学“圣克里门特·奥赫里德斯基”）

We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 1: Check-Worthiness. The task asks to predict which claims in a political debate should be prioritized for fact-checking. In particular, given a debate or a political speech, the goal was to produce a ranked list of its sentences based on their worthiness for fact checking. We offered the task in both English and Arabic, based on debates from the 2016 US Presidential Campaign, as well as on some speeches during and after the campaign. A total of 30 teams r…

2018-08-08cs.CL

It Takes Nine to Smell a Rat: Neural Multi-Task Learning for Check-Worthiness Prediction

Sofia University（索非亚大学）；University of Copenhagen（哥本哈根大学）；Amazon Core ML（亚马逊核心机器学习部门）；Università di Bologna（博洛尼亚大学）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）

We propose a multi-task deep-learning approach for estimating the check-worthiness of claims in political debates. Given a political debate, such as the 2016 US Presidential and Vice-Presidential ones, the task is to predict which statements in the debate should be prioritized for fact-checking. While different fact-checking organizations would naturally make different choices when analyzing the same debate, we show that it pays to learn from multiple sources simultaneously (PolitiFact, FactCheck, ABC, CNN, NPR, NYT, Chicago Tribune, The Guardian, and Washington Post) in a multi-task learning …

2019-08-19cs.AIcs.CL

Claim Check-Worthiness Detection as Positive Unlabelled Learning

University of Copenhagen（哥本哈根大学）

As the first step of automatic fact checking, claim check-worthiness detection is a critical component of fact checking systems. There are multiple lines of research which study this problem: check-worthiness ranking from political speeches and debates, rumour detection on Twitter, and citation needed detection from Wikipedia. To date, there has been no structured comparison of these various tasks to understand their relatedness, and no investigation into whether or not a unified approach to all of them is achievable. In this work, we illuminate a central challenge in claim check-worthiness de…

2020-09-16cs.CLcs.LG

Fraunhofer SIT 在 CheckThat! 2023 中的参与：以 Check-Worthiness 分类为例，利用 Model Souping 应对分类不确定性

Fraunhofer SIT at CheckThat! 2023: Tackling Classification Uncertainty Using Model Souping on the Example of Check-Worthiness Classification

Fraunhofer Institute for Secure Information Technology SIT（弗劳恩霍夫安全信息技术研究所）；ATHENE - National Research Center for Applied Cybersecurity（ATHENE——国家应用网络安全研究中心）

Fraunhofer SIT 团队在 CLEF-2023 CheckThat! Task 1B 中提出基于 Model Souping 的集成分类方法，用于政治辩论文本的值得核查性分类，在英语数据集上取得 0.878 的 F1 分数，排名第二。

2023-07-27cs.CLcs.LG

IAI团队在CheckThat! 2024：用于可核查声明检测的Transformer模型与数据增强

IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection

University of Stavanger（斯塔万格大学）

本文介绍IAI团队在CheckThat! 2024任务1中，利用Transformer模型、少样本链式思维、微调、数据增强和跨语言迁移学习，在英荷阿三语可核查声明检测中分别取得第九、第三和第一的成绩。

2024-08-02cs.CL

RAVE：检索与评分感知的可验证声明检测

RAVE: Retrieval and Scoring Aware Verifiable Claim Detection

Queen Mary University of London（伦敦女王学院）

本文提出RAVE框架，通过结合证据检索与相关性及来源可信度信号来检测可验证声明，在CT22-test和PoliClaim-test上均优于文本和检索基线方法。

2025-09-19cs.CL

更多相关论文收起更多论文

FactFinders at CheckThat! 2024: 通过数据剪枝利用大型语言模型优化值得核查的陈述检测

FactFinders at CheckThat! 2024: Refining Check-worthy Statement Detection with LLMs through Data Pruning

School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦大学玛丽女王学院电子工程与计算机科学学院）

本研究利用八种开源大语言模型结合微调和提示工程，通过两步数据剪枝方法，在CheckThat! 2024英语任务中高效识别值得核查的陈述，仅用44%训练数据即获第一名。

2024-06-26cs.CL

Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media

Università di Bologna（博洛尼亚大学）；Qatar University（卡塔尔大学）；Qatar Computing Research Institute, HBKU（卡塔尔计算研究所（哈马德·本·哈利法大学））；Sofia University “St Kliment Ohridski”（索菲亚大学“圣克莱门特·奥赫里德斯基”）

We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in two different languages: English and Arabic. The first four tasks compose the full pipeline of claim verification in social media: Task 1 on check-worthiness estimation, Task 2 on retrieving previously fact-checked claims, Task 3 on evidence retrieval, and Task 4 on claim verification. The lab is completed with Task 5 on check-worthiness estimation in political debates and speeches. A total of 67 teams registered to participate in the lab (up from 47 at CLEF 2019), and 23 of them actu…

2020-07-15cs.CLcs.IRcs.LG

ClaimRank: Detecting Check-Worthy Claims in Arabic and English

Qatar Computing Research Institute, HBKU, Qatar（卡塔尔计算研究所（HBKU））；Sofia University “St. Kliment Ohridski”, Bulgaria（索非亚大学“圣克莱门特·奥赫里德斯基”）；Amazon, Barcelona, Spain（亚马逊（巴塞罗那））

We present ClaimRank, an online system for detecting check-worthy claims. While originally trained on political debates, the system can work for any kind of text, e.g., interviews or regular news articles. Its aim is to facilitate manual fact-checking efforts by prioritizing the claims that fact-checkers should consider first. ClaimRank supports both Arabic and English, it is trained on actual annotations from nine reputable fact-checking organizations (PolitiFact, FactCheck, ABC, CNN, NPR, NYT, Chicago Tribune, The Guardian, and Washington Post), and thus it can mimic the claim selection stra…

2018-04-20cs.CL

Automatic Fact-Checking Using Context and Discourse Information

We study the problem of automatic fact-checking, paying special attention to the impact of contextual and discourse information. We address two related tasks: (i) detecting check-worthy claims, and (ii) fact-checking claims. We develop supervised systems based on neural networks, kernel-based support vector machines, and combinations thereof, which make use of rich input representations in terms of discourse cues and contextual features. For the check-worthiness estimation task, we focus on political debates, and we model the target claim in the context of the full intervention of a participan…

2019-08-04cs.AIcs.CL

Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of Claims

Computer Science and Engineering Department, Qatar University（卡塔尔大学计算机科学与工程系）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）；DIT, Università di Bologna（博洛尼亚大学DIT（此处DIT按原文保留，若需全称需更多上下文））；Department of Computer Science, University of Copenhagen（哥本哈根大学计算机科学系）

We present an overview of the second edition of the CheckThat! Lab at CLEF 2019. The lab featured two tasks in two different languages: English and Arabic. Task 1 (English) challenged the participating systems to predict which claims in a political debate or speech should be prioritized for fact-checking. Task 2 (Arabic) asked to (A) rank a given set of Web pages with respect to a check-worthy claim based on their usefulness for fact-checking that claim, (B) classify these same Web pages according to their degree of usefulness for fact-checking the target claim, (C) identify useful passages fr…

2021-09-25cs.AIcs.CLcs.IR