Assessing the Impact of Conspiracy Theories Using Large Language Models
Arizona State University(亚利桑那州立大学);University of Washington(华盛顿大学);USC Information Sciences Institute(南加州大学信息科学研究所);ASU Institute for Social Science Research(亚利桑那州立大学社会科学研究所)
Bohan Jiang, Dawei Li, Zhen Tan, Xinyi Zhou, Ashwin Rao, Kristina Lerman, H. Russell Bernard, Huan Liu
Measuring the relative impact of CTs is important for prioritizing responses and allocating resources effectively, especially during crises. However, assessing the actual impact of CTs on the public poses unique challenges. It requires not only the collection of CT-specific knowledge but also diverse information from social, psychological, and cultural dimensions. Recent advancements in large language models (LLMs) suggest their potential utility in this context, not only due to their extensive knowledge from large training corpora but also because they can be harnessed for complex reasoning. In this work, we develop datasets of popular CTs with human-annotated impacts. Borrowing insights from human impact assessment processes, we then design tailored strategies to leverage LLMs for performing human-like CT impact assessments. Through rigorous experiments, we textit{discover that an impact assessment mode using multi-step reasoning to analyze more CT-related evidence critically produces accurate results; and most LLMs demonstrate strong bias, such as assigning higher impacts to CTs presented earlier in the prompt, while generating less accurate impact assessments for emotionally charged and verbose CTs.
ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety
Technical University of Darmstadt(达姆施塔特工业大学);National Research Center for Applied Cybersecurity ATHENE(国家应用网络安全研究中心ATHENE);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Walmart Inc.(沃尔玛公司)
Codes, Patterns and Shapes of Contemporary Online Antisemitism and Conspiracy Narratives -- an Annotation Guide and Labeled German-Language Dataset in the Context of COVID-19
Hochschule für Technik und Wirtschaft Berlin(柏林工程与经济应用技术大学);Alice Salomon Hochschule Berlin(柏林爱丽丝·萨洛蒙应用科学大学)