Instituto de Ciencias de la Computación (UBA-CONICET)(计算机科学研究所(UBA-CONICET));Hospital Dr. Lucio Melendez(Lucio Melendez医院);Universidad de Buenos Aires(布宜诺斯艾利斯大学);Universidad Católica de Chile(智利天主大学);MICS, CentraleSupélec - Université Paris-Saclay(MICS,CentraleSupélec-巴黎萨克雷大学);Universidad Nacional de Córdoba(科迪利亚国家大学);Fundación Vía Libre(自由之路基金会);CIECTI
Large language models (LLMs) have been shown to exhibit biases against LGBTQ+ populations. While safety training may lessen explicit expressions of bias, previous work has shown that implicit stereotype-driven associations often persist. In this work, we examine implicit bias toward transgender people in two main scenarios. First, we adapt word association tests to measure whether LLMs disproportionately pair negative concepts with "transgender" and positive concepts with "cisgender". Second, acknowledging the well-documented systemic challenges that transgender people encounter in real-world healthcare settings, we examine implicit biases that may emerge when LLMs are applied to healthcare decision-making. To this end, we design a healthcare appointment allocation task where models act as scheduling agents choosing between cisgender and transgender candidates across medical specialties prone to stereotyping. We evaluate seven LLMs in English and Spanish. Our results show consistent bias in categories such as appearance, risk, and veracity, indicating stronger negative associations with transgender individuals. In the allocation task, transgender candidates are favored for STI and mental health services, while cisgender candidates are preferred in gynecology and breast care. These findings underscore the need for research that address subtle stereotype-driven biases in LLMs to ensure equitable treatment of transgender people in healthcare applications.
A word association network methodology for evaluating implicit biases in LLMs compared to humans
University of Pisa, Department of Computer Science(比萨大学计算机科学系);National Research Council of Italy, Institute of Information Science and Technologies(意大利国家研究委员会信息科学与技术研究所);University of Trento, Department of Psychology and Cognitive Science(特伦托大学心理学与认知科学系)
Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院人工智能安全国家重点实验室,计算技术研究所);Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院网络数据科学与技术重点实验室,计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
Measuring Implicit Bias Using SHAP Feature Importance and Fuzzy Cognitive Maps
Eindhoven University of Technology(埃因霍温理工大学);Eindhoven Artificial Intelligence Systems Institute(埃因霍温人工智能系统研究院);Tilburg University(蒂尔堡大学);Hasselt University(哈塞尔特大学)