Misinformation emerges in times of uncertainty when credible information is limited. This is challenging for NLP-based fact-checking as it relies on counter-evidence, which may not yet be available. Despite increasing interest in automatic fact-checking, it is still unclear if automated approaches can realistically refute harmful real-world misinformation. Here, we contrast and compare NLP fact-checking with how professional fact-checkers combat misinformation in the absence of counter-evidence. In our analysis, we show that, by design, existing NLP task definitions for fact-checking cannot refute misinformation as professional fact-checkers do for the majority of claims. We then define two requirements that the evidence in datasets must fulfill for realistic fact-checking: It must be (1) sufficient to refute the claim and (2) not leaked from existing fact-checking articles. We survey existing fact-checking datasets and find that all of them fail to satisfy both criteria. Finally, we perform experiments to demonstrate that models trained on a large-scale fact-checking dataset rely on leaked evidence, which makes them unsuitable in real-world scenarios. Taken together, we show that current NLP fact-checking cannot realistically combat real-world misinformation because it depends on unrealistic assumptions about counter-evidence in the data.
CANDY: Benchmarking LLMs' Limitations and Assistive Potential in Chinese Misinformation Fact-Checking
School of Cyber Science and Engineering, Sichuan University, China(四川大学网络科学与工程学院);College of Computer Science, Sichuan University, China(四川大学计算机学院);Institute of Data Science, National University of Singapore, Singapore(新加坡国立大学数据科学研究所)
HiEAG: Evidence-Augmented Generation for Out-of-Context Misinformation Detection
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Institute of Artificial Intelligence, Soochow University(苏州大学人工智能研究院);School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院)
Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs
School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算与信息系统学院);Institute of Operations Research and Analytics, National University of Singapore(新加坡国立大学运筹学与分析研究所)
"Image, Tell me your story!" Predicting the original meta-context of visual misinformation
TU Darmstadt(达姆施塔特工业大学);KU Leuven(鲁汶大学);Hessian Center for AI (hessian.AI)(黑森人工智能中心);Ubiquitous Knowledge Processing Lab (UKP Lab)(普适知识处理实验室);Department of Computer Science(计算机科学系);Department of Electrical Engineering(电气工程系)
Mila;McGill University(麦吉尔大学);Université de Montréal(蒙特利尔大学);Nord AI(北AI);Harvard University(哈佛大学);Supervised Program for Alignment Research (SPAR)(对齐研究监督计划)
The Psychology of Falsehood: A Human-Centric Survey of Misinformation Detection
Indian Institute of Technology Delhi(印度理工学院德里);Max Planck Institute for Security and Privacy(安全与隐私马克斯·普朗克研究所);University of Minnesota - Twin Cities(明尼苏达大学双城分校)
MIC: Explaining Image-Claim Inconsistencies in AI-Generated Multimodal Misinformation
KU Leuven(鲁汶大学);TU Darmstadt(达姆施塔特工业大学);National Research Center for Applied Cybersecurity ATHENE(国家应用网络安全研究中心ATHENE);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学先进交叉学科学院);State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院)
Combating Misinformation in the Arab World: Challenges & Opportunities
New York University Abu Dhabi(纽约大学阿布扎赫尔分校);Qatar Computing Research Institute(卡塔尔计算研究所);College of Science and Engineering, Hamad Bin Khalifa University(哈马德·本·卡西姆大学科学与工程学院);EURECOM Biot France(EURECOM法国生物研究中心)
TrumorGPT: Graph-Based Retrieval-Augmented Large Language Model for Fact-Checking
Yam Pak Charitable Foundation School of Computing and Information Sciences, Saint Francis University, Hong Kong(Yam Pak Charitable Foundation 计算与信息科学学院,圣弗朗西斯大学,香港);Department of Applied Mathematics, Chung Yuan Christian University(应用数学系, Chung Yuan Christian University);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)
Grounding Fallacies Misrepresenting Scientific Publications in Evidence
Ubiquitous Knowledge Processing Lab (UKP Lab)(通用知识处理实验室(UKP实验室));TU Darmstadt(达姆施塔特工业大学);Hessian Center for AI (hessian.AI)(黑森人工智能中心);IT:U Interdisciplinary Transformation University Austria(奥地利跨学科转型大学(IT:U));IBM Research Ireland(IBM爱尔兰研究院);MBZUAI(穆罕默德·本·扎耶德人工智能大学)