Studies have shown that some Natural Language Processing (NLP) systems encode and replicate harmful biases with potential adverse ethical effects in our society. In this article, we propose an approach for identifying gender and racial stereotypes in word embeddings trained on judicial opinions from U.S. case law. Embeddings containing stereotype information may cause harm when used by downstream systems for classification, information extraction, question answering, or other machine learning systems used to build legal research tools. We first explain how previously proposed methods for identifying these biases are not well suited for use with word embeddings trained on legal opinion text. We then propose a domain adapted method for identifying gender and racial biases in the legal domain. Our analyses using these methods suggest that racial and gender biases are encoded into word embeddings trained on legal opinions. These biases are not mitigated by exclusion of historical data, and appear across multiple large topical areas of the law. Implications for downstream systems that use legal opinion word embeddings and suggestions for potential mitigation strategies based on our observations are also discussed.
As NLP models become more integrated with the everyday lives of people, it becomes important to examine the social effect that the usage of these systems has. While these models understand language and have increased accuracy on difficult downstream tasks, there is evidence that these models amplify gender, racial and cultural stereotypes and lead to a vicious cycle in many settings. In this survey, we analyze the origins of biases, the definitions of fairness, and how different subfields of NLP mitigate bias. We finally discuss how future studies can work towards eradicating pernicious biases…
Seeds of Stereotypes: A Large-Scale Textual Analysis of Race and Gender Associations with Diseases in Online Sources
Aarhus University（奥胡斯大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Alberta（阿尔伯塔大学）；University of Manitoba（曼尼托巴大学）；Mbarara University of Science and Technology（姆巴拉拉科技大学）；Fundacion Arturo Lopez Perez（阿图罗·洛佩斯·佩雷斯基金会）；Mass General Brigham（麻省总医院布里格姆）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Emory University School of Medicine（埃默里大学医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）
本文通过分析 Arxiv、Wikipedia 和 Common Crawl 中18种疾病与种族、性别术语的共现，并对比患病率与 GPT-4 输出，揭示在线文本及 LLM 预训练数据中的人口统计学偏见。
Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts
School of Computer Science and Technology, East China Normal University（东华师范大学计算机科学与技术学院）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University（东华师范大学教育人工智能研究所）；Faculty of Education, East China Normal University（东华师范大学教育学院）
Contextual word embeddings such as BERT have achieved state of the art performance in numerous NLP tasks. Since they are optimized to capture the statistical properties of training data, they tend to pick up on and amplify social stereotypes present in the data as well. In this study, we (1)~propose a template-based method to quantify bias in BERT; (2)~show that this method obtains more consistent results in capturing social biases than the traditional cosine based method; and (3)~conduct a case study, evaluating gender bias in a downstream task of Gender Pronoun Resolution. Although our case …
