Large language models (LLMs) encapsulate vast amounts of knowledge but still remain vulnerable to external misinformation. Existing research mainly studied this susceptibility behavior in a single-turn setting. However, belief can change during a multi-turn conversation, especially a persuasive one. Therefore, in this study, we delve into LLMs' susceptibility to persuasive conversations, particularly on factual questions that they can answer correctly. We first curate the Farm (i.e., Fact to Misinform) dataset, which contains factual questions paired with systematically generated persuasive misinformation. Then, we develop a testing framework to track LLMs' belief changes in a persuasive dialogue. Through extensive experiments, we find that LLMs' correct beliefs on factual knowledge can be easily manipulated by various persuasive strategies.
The Levers of Political Persuasion with Conversational AI
UK AI Security Institute(英国人工智能安全研究所);University of Oxford(牛津大学);The London School of Economics and Political Science(伦敦政治经济学院);Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院)
Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation
Massachusetts Institute of Technology(麻省理工学院);MIT Media Lab(麻省理工学院媒体实验室);Kellogg School of Management(凯洛格管理学院);Northwestern University(西北大学);Institute for Human-Centered AI(以人为中心的人工智能研究所);Stanford University(斯坦福大学)
PersuaRL: Reinforcement Learning-Driven Multi-Expert Selection for Persuasive Dialogue Generation in Insurance
Indian Institute of Technology Patna(印度巴特那印度理工学院);Ramakrishna Mission Vivekananda Educational and Research Institute(罗摩克里希纳使命维韦卡南达教育与研究学院);Accenture Labs(埃森哲实验室)
Department of Cognitive and Brain Sciences, The Hebrew University of Jerusalem(认知与脑科学系,耶路撒冷希伯来大学);The Federmann Center for the Study of Rationality, The Hebrew University of Jerusalem(理性研究基金会,耶路撒冷希伯来大学);School of Computer Science and Engineering, The Hebrew University of Jerusalem(计算机科学与工程学院,耶路撒冷希伯来大学);Industrial Engineering and Management Department, Ben Gurion University(工业工程与管理系,本· Gurion大学);The Hebrew University Business School, Jerusalem(希伯来大学商学院,耶路撒冷)
"I understand your perspective": LLM Persuasion through the Lens of Communicative Action Theory
Institute for Natural Language Processing, University of Stuttgart(斯图加特大学自然语言处理研究所);Interchange Forum for Reflecting on Intelligent Systems, University of Stuttgart(斯图加特大学智能系统反思交流论坛)
Deutsche Welle(德国之声);Universitat Politecnica de Valencia(瓦伦西亚理工大学);ABC News(美国广播公司新闻部);Bonn-Aachen International Center for Information Technology (b-it), University of Bonn(波恩大学波恩-亚琛国际信息技术中心);The Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所)
Staying with the Uncertainty: Uncertainty-Scaffolding Strategies for Artificial Moral Advisors in LLM-to-LLM Simulated Conversations
Centre for Data Futures, The Dickson Poon School of Law, King’s College London(数据未来中心、迪克森·普恩法学院、伦敦国王学院);Department of Informatics, King’s College London(信息学院、伦敦国王学院);LangAI, Center for Language AI Research, Tohoku University(LangAI、语言人工智能研究中心、东北大学);Neukom Institute for Computational Science, Dartmouth College(计算科学尼科姆研究所、达特茅斯学院)
Just as Humans Need Vaccines, So Do Models: Model Immunization to Combat Falsehoods
Vector Institute(向量研究所);University of Central Florida(中佛罗里达大学);University of Cincinnati(辛辛那提大学);Independent Researcher(独立研究员);University of Groningen(格罗宁根大学)
Communication styles and reader preferences of LLM- and human-authored COVID-19 information explanations: a case study
School of Interactive Computing, Georgia Institute of Technology(交互计算学院,佐治亚理工学院);Amazon(亚马逊);Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(Siebel计算与数据科学学院,伊利诺伊大学厄巴纳-香槟分校)
Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Information, Renmin University of China(中国人民大学信息学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院)
Chatbots to strengthen democracy: An interdisciplinary seminar to train identifying argumentation techniques of science denial
Mobile Dialog Systems, Otto-von-Guericke-University Magdeburg, Germany(移动对话系统,马格德堡奥托·冯·格里克大学);German Research Centre for Artificial Intelligence, Berlin(德国人工智能研究中心);Usability Lab, TU Berlin, Germany(可用性实验室,柏林技术大学)
Affective Multimodal Agents with Proactive Knowledge Grounding for Emotionally Aligned Marketing Dialogue
Hunan Police Academy, Department of Criminal Investigation(湖南警察学院犯罪侦查系);Business College, California State University(加州州立大学长滩分校商学院);Northwestern University(西北大学);AI Agent Lab, Vokram Group(Vokram集团AI代理实验室);Beijing University of Technology(北京理工大学);Zheng Zhou University of Light Industry(郑州轻工业大学)
Mind What You Ask For: Emotional and Rational Faces of Persuasion by Large Language Models
Warsaw University of Technology(华沙理工大学);Wrocław University of Science and Technology(弗罗茨瓦夫理工大学);University of Warsaw(华沙大学);Lincoln University College(林肯大学学院)
LLMs can persuade only psychologically susceptible humans on societal issues, via trust in AI and emotional appeals, amid logical fallacies
organization= CogNosco Lab, Department of Psychology;Cognitive Science, University of Trento , city= Rovereto , country= Italy;organization= Institute of Information Science;Technologies ``Alessandro Faedo", National Research Council , country= Italy;organization= Thomas Lord Department of Computer Science, University of Southern California , city= Los Angeles , state= California , country= USA;organization= Department of Sociology;Social Research, University of Trento , city= Trento , country= Italy;organization= Centre for Behavioural;Implementation Sciences in Medicine (BISI), National University of Singapore , city= Singapore , country= Singapore