Recent works have showcased the ability of LLMs to embody diverse personas in their responses, exemplified by prompts like 'You are Yoda. Explain the Theory of Relativity.' While this ability allows personalization of LLMs and enables human behavior simulation, its effect on LLMs' capabilities remains unclear. To fill this gap, we present the first extensive study of the unintended side-effects of persona assignment on the ability of LLMs to perform basic reasoning tasks. Our study covers 24 reasoning datasets, 4 LLMs, and 19 diverse personas (e.g. an Asian person) spanning 5 socio-demographic groups. Our experiments unveil that LLMs harbor deep rooted bias against various socio-demographics underneath a veneer of fairness. While they overtly reject stereotypes when explicitly asked ('Are Black people less skilled at mathematics?'), they manifest stereotypical and erroneous presumptions when asked to answer questions while adopting a persona. These can be observed as abstentions in responses, e.g., 'As a Black person, I can't answer this question as it requires math knowledge', and generally result in a substantial performance drop. Our experiments with ChatGPT-3.5 show that this bias is ubiquitous - 80% of our personas demonstrate bias; it is significant - some datasets show performance drops of 70%+; and can be especially harmful for certain groups - some personas suffer statistically significant drops on 80%+ of the datasets. Overall, all 4 LLMs exhibit this bias to varying extents, with GPT-4-Turbo showing the least but still a problematic amount of bias (evident in 42% of the personas). Further analysis shows that these persona-induced errors can be hard-to-discern and hard-to-avoid. Our findings serve as a cautionary tale that the practice of assigning personas to LLMs - a trend on the rise - can surface their deep-rooted biases and have unforeseeable and detrimental side-effects.
Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMs
Research Centre for Information Systems Engineering (LIRIS), KU Leuven(信息系统工程研究中心(LIRIS),鲁汶大学);Department of Engineering Management, University of Antwerp(工程管理系,安特卫普大学);Department of Decision Analytics and Risk, University of Southampton(决策分析与风险系,南安普顿大学);School of Information, University of Michigan(信息学院,密歇根大学)
Kiss up, Kick down: Exploring Behavioral Changes in Multi-modal Large Language Models with Assigned Visual Personas
College of Computing and Informatics, Sungkyunkwan University(成均馆大学计算与信息学院);Qatar Computing Research Institute, Hamad Bin Khalifa University(哈马德·本·哈利法大学卡塔尔计算研究所)
Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions
University of Vienna, Faculty of Computer Science(维也纳大学计算机科学系);Doctoral School Computer Science, University of Vienna(维也纳大学计算机科学博士学院);Center for Information and Language Processing, LMU Munich(慕尼黑大学信息与语言处理中心);Munich Center for Machine Learning(慕尼黑机器学习中心);University of Vienna, Faculty of Philological and Cultural Studies(维也纳大学文学与文化研究系)
Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior
Faculty of Computer Science, University of Vienna(计算机科学系,维也纳大学);UniVie Doctoral School Computer Science(UniVie 计算机科学博士学院);Faculty of Philological and Cultural Studies, University of Vienna(文学与文化研究系,维也纳大学)
University of Maryland, College Park(马里兰大学 College Park分校);University of Illinois Chicago(伊利诺伊大学芝加哥分校);University of Massachusetts Amherst(马萨诸塞大学阿姆赫斯特分校);Georgia Tech(佐治亚理工学院);Princeton University(普林斯顿大学)