Despite the many use cases for large language models (LLMs) in creating personalized chatbots, there has been limited research on evaluating the extent to which the behaviors of personalized LLMs accurately and consistently reflect specific personality traits. We consider studying the behavior of LLM-based agents which we refer to as LLM personas and present a case study with GPT-3.5 and GPT-4 to investigate whether LLMs can generate content that aligns with their assigned personality profiles. To this end, we simulate distinct LLM personas based on the Big Five personality model, have them complete the 44-item Big Five Inventory (BFI) personality test and a story writing task, and then assess their essays with automatic and human evaluations. Results show that LLM personas' self-reported BFI scores are consistent with their designated personality types, with large effect sizes observed across five traits. Additionally, LLM personas' writings have emerging representative linguistic patterns for personality traits when compared with a human writing corpus. Furthermore, human evaluation shows that humans can perceive some personality traits with an accuracy of up to 80%. Interestingly, the accuracy drops significantly when the annotators were informed of AI authorship.
Exploring the Impact of Personality Traits on Conversational Recommender Systems: A Simulation with Large Language Models
Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；Singapore Management University（新加坡管理大学）；Hong Kong Baptist University（香港浸会大学）；Huazhong University of Science and Technology（华中科技大学）
Do Personality Traits Interfere? Geometric Limitations of Steering in Large Language Models
Network Analysis and Social Influence Modelling (NASIM) Lab（网络分析与社会影响建模实验室）；School of Physics Maths and Computing（物理数学计算学院）；The University of Western Australia（西澳大学）；School of Computing（计算学院）；Macquarie University（麦觉瑞大学）
Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnaires
School of Physics, Mathematics and Computing（物理、数学与计算学院）；University of Western Australia（西澳大学）；School of Computing（计算机学院）；School of Psychological Sciences（心理科学学院）
Neuron-based Personality Trait Induction in Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院）；Tongyi Lab（通义实验室）；Institute of Statistics and Big Data, Renmin University of China（中国人民大学统计与大数据研究院）
Personality Vector: Modulating Personality of Large Language Models by Model Merging
Department of Human-Artificial Intelligence Interaction, Sungkyunkwan University（人类-人工智能交互系，成均馆大学）；Department of Immersive Media Engineering, Sungkyunkwan University（沉浸式媒体工程系，成均馆大学）
Predicting the Big Five Personality Traits in Chinese Counselling Dialogues Using Large Language Models
Zhejiang University（浙江大学）；School of Engineering, Westlake University（西湖大学工程学院）；Westlake University（西湖大学）；School of Early Childhood Education, Hangzhou Polytechnic（杭州职业技术学院幼儿教育学院）
The Dark Patterns of Personalized Persuasion in Large Language Models: Exposing Persuasive Linguistic Features for Big Five Personality Traits in LLMs Responses
Warsaw University of Technology（华沙理工大学）；University of Warsaw（华沙大学）
