Persona assignment has become a common strategy for customizing LLM use to particular tasks and contexts. In this study, we explore how evaluation of different nations change when LLMs are assigned specific nationality personas. We assign 193 different nationality personas (e.g., an American person) to four LLMs and examine how the LLM evaluations (or ''perceptions'')of countries change. We find that all LLM-persona combinations tend to favor Western European nations, though nation-personas push LLM behaviors to focus more on and treat the nation-persona's own region more favorably. Eastern European, Latin American, and African nations are treated more negatively by different nationality personas. We additionally find that evaluations by nation-persona LLMs of other nations correlate with human survey responses but fail to match the values closely. Our study provides insight into how biases and stereotypes are realized within LLMs when adopting different national personas. In line with the ''Blueprint for an AI Bill of Rights'', our findings underscore the critical need for developing mechanisms to ensure that LLM outputs promote fairness and avoid over-generalization.
Nationality encoding in language model hidden states: Probing culturally differentiated representations in persona-conditioned academic text
Language Centre, School of Language, Literature, Music and Visual Culture, University of Aberdeen（语言中心，语言文学音乐与视觉文化学院，阿伯丁大学）；School of Natural and Computing Sciences, University of Aberdeen（自然与计算科学学院，阿伯丁大学）；School of Language, Literature, Music and Visual Culture, University of Aberdeen（语言文学音乐与视觉文化学院，阿伯丁大学）
Language-Agnostic Bias Detection in Language Models with Bias Probing
Center for Information and Language Processing, LMU Munich（慕尼黑大学信息与语言处理中心）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Data Analytics and Computational Social Science, University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校数据分析与计算社会科学系）；Marmara University（马尔马拉大学）；Language Technology Lab, University of Cambridge（剑桥大学语言技术实验室）；Harvard Kennedy School（哈佛大学肯尼迪学院）
Where Should I Study? Biased Language Models Decide! Evaluating Fairness in LMs for Academic Recommendations
Centre for Responsible AI (CeRAI), Wadhwani School of Data Science and AI (WSAI)（负责任人工智能中心（CeRAI）、Wadhwani数据科学与人工智能学校（WSAI））；Indian Institute of Technology Madras（印度理工学院马德拉斯学院）
Source framing triggers systematic evaluation bias in Large Language Models
Institute of Biomedical Ethics and History of Medicine, University of Zurich（苏黎世大学生物医学伦理与医学史研究所）；Center for Medical Ethics, University of Oslo（奥斯陆大学医学伦理中心）
