With the wide-scale adoption of conversational AI systems, AI are now able to exert unprecedented influence on human opinion and beliefs. Recent work has shown that many Large Language Models (LLMs) comply with requests to persuade users into harmful beliefs or actions when prompted and that model persuasiveness increases with model scale. However, this prior work looked at persuasion from the threat model of $\textit{misuse}$ (i.e., a bad actor asking an LLM to persuade). In this paper, we instead aim to answer the following question: Under what circumstances would models persuade $\textit{without being explicitly prompted}$, which would shape how concerned we should be about such emergent persuasion risks. To achieve this, we study unprompted persuasion under two scenarios: (i) when the model is steered (through internal activation steering) along persona traits, and (ii) when the model is supervised-finetuned (SFT) to exhibit the same traits. We showed that steering towards traits, both related to persuasion and unrelated, does not reliably increase models' tendency to persuade unprompted, however, SFT does. Moreover, SFT on general persuasion datasets containing solely benign topics admits a model that has a higher propensity to persuade on controversial and harmful topics--showing that emergent harmful persuasion can arise and should be studied further.
It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics
Université de Montréal, MILA(蒙特利尔大学,MILA);Carnegie Mellon University(卡内基梅隆大学);MIT, Center for Research and Teaching in Economics(麻省理工学院,经济研究与教学中心);Cornell University, University of Regina(康奈尔大学, Regina大学);Cornell University, MIT(康奈尔大学,麻省理工学院)
Assessing and Explaining the Persuadability of Large Language Models as Legal Decision Tools
School of Law;Criminology Maynooth University Maynooth Ireland;School of Computer Science University College Dublin Dublin Ireland;Criminology Maynooth University;School of Computer Science University College Dublin
IT University of Copenhagen(哥本哈根信息技术大学);University of Pisa(比萨大学);City St. George’s, University of London(伦敦城市圣乔治大学);Universitat Politècnica de Catalunya(加泰罗尼亚理工大学)