Recent large language models (LLMs) have been shown to be effective for misinformation detection. However, the choice of LLMs for experiments varies widely, leading to uncertain conclusions. In particular, GPT-4 is known to be strong in this domain, but it is closed source, potentially expensive, and can show instability between different versions. Meanwhile, alternative LLMs have given mixed results. In this work, we show that Zephyr-7b presents a consistently viable alternative, overcoming key limitations of commonly used approaches like Llama-2 and GPT-3.5. This provides the research community with a solid open-source option and shows open-source models are gradually catching up on this task. We then highlight how GPT-3.5 exhibits unstable performance, such that this very widely used model could provide misleading results in misinformation detection. Finally, we validate new tools including approaches to structured output and the latest version of GPT-4 (Turbo), showing they do not compromise performance, thus unlocking them for future research and potentially enabling more complex pipelines for misinformation mitigation.
Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4
McGill University(麦吉尔大学);Mila(米拉研究所);Université de Montréal(蒙特利尔大学);Eindhoven University of Technology(埃因霍温理工大学);University of Pennsylvania(宾夕法尼亚大学);European University Institute(欧洲大学研究院)
Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences(中国科学院大学先进交叉学科学院);State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院)
A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Harbin Institute of Technology(哈尔滨工业大学);Ant Group, Alibaba(蚂蚁集团,阿里巴巴);Northeast Forest University(东北林业大学)
Optimizing Social Media Annotation of HPV Vaccine Skepticism and Misinformation Using Large Language Models: An Experimental Evaluation of In-Context Learning and Fine-Tuning Stance Detection Across Multiple Models
School of Journalism and Mass Communication, University of Wisconsin–Madison(威斯康星大学麦迪逊分校新闻与大众传播学院);Department of Electrical & Computer Engineering, University of Wisconsin–Madison(威斯康星大学麦迪逊分校电气与计算机工程系);Department of Computer Science, University of Wisconsin–Madison(威斯康星大学麦迪逊分校计算机科学系);Department of Psychology, University of Wisconsin–Madison(威斯康星大学麦迪逊分校心理学系);School of Interactive Computing, Georgia Institute of Technology(佐治亚理工学院交互计算学院);Institute of People and Technology, Georgia Institute of Technology(佐治亚理工学院人与技术研究所)
Is Explanation the Cure? Misinformation Mitigation in the Short Term and Long Term
Institute of Information Science, Academia Sinica(中央研究院资讯科学研究所);Department of Computer Science, National Tsing Hua University(国立清华大学资讯工程学系);The University of Texas at Austin(德克萨斯大学奥斯汀分校);The Pennsylvania State University(宾夕法尼亚州立大学)
Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs
School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算与信息系统学院);Institute of Operations Research and Analytics, National University of Singapore(新加坡国立大学运筹学与分析研究所)
Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability
GESIS – Leibniz Institute for the Social Sciences(GESIS社会科学院研究所);RWTH Aachen University(亚琛RWTH大学);University of Graz(格拉茨大学);Complexity Science Hub(复杂性科学中心)
On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey
Institute for Clarity in Documentation(清晰文档研究所);Inria Paris-Rocquencourt(巴黎- Rocquencourt 国家信息与自动化研究所);Rajiv Gandhi University(拉贾·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒尔研究实验室);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Desert Camels and Oil Sheikhs: Arab-Centric Red Teaming of Frontier LLMs
MBZUAI(穆罕默德·本·扎耶德人工智能大学);University of Khartoum(喀土穆大学);Universität des Saarlandes(萨尔大学);University of British Columbia(不列颠哥伦比亚大学);Invertible AI(可逆人工智能公司);Vector Institute(向量研究所)
Gender Bias in LLM Hiring Decisions: Evidence from a Japanese Context and Evaluation of Mitigation Strategies
Shibaura Institute of Technology, Tokyo, Japan(Shibaura技术学院,东京,日本);Amsterdam University of Applied Sciences, Amsterdam, Netherlands(阿姆斯特丹应用科学大学,阿姆斯特丹,荷兰);University of Pennsylvania, Philadelphia, USA(宾夕法尼亚大学,费城,美国);Carnegie Mellon University, Pittsburgh, USA(卡内基梅隆大学,匹兹堡,美国);Keio University, Tokyo, Japan(庆应大学,东京,日本)
Susceptibility of Large Language Models to User-Driven Factors in Medical Queries
Yonsei University College of Medicine(延世大学医学院);Yonsei University(延世大学);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Oncosoft Inc.(Oncosoft公司)