arXiv 2405.050492024-05-08cs.CL

刻板印象的种子：在线来源中疾病与种族和性别关联的大规模文本分析

Seeds of Stereotypes: A Large-Scale Textual Analysis of Race and Gender Associations with Diseases in Online Sources

Aarhus University（奥胡斯大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Alberta（阿尔伯塔大学）；University of Manitoba（曼尼托巴大学）；Mbarara University of Science and Technology（姆巴拉拉科技大学）；Fundacion Arturo Lopez Perez（阿图罗·洛佩斯·佩雷斯基金会）；Mass General Brigham（麻省总医院布里格姆）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Emory University School of Medicine（埃默里大学医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）

Lasse Hyldig Hansen, Nikolaj Andersen, Jack Gallifant, Liam G. McCoy, James K Stone, Nura Izath, Marcela Aguirre-Jerez, Danielle S Bitterman, Judy Gichoya, Leo Anthony Celi

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文通过分析 Arxiv、Wikipedia 和 Common Crawl 中18种疾病与种族、性别术语的共现，并对比患病率与 GPT-4 输出，揭示在线文本及 LLM 预训练数据中的人口统计学偏见。

摘要：

背景：大型语言模型（LLM）的进步在医疗保健领域具有变革性潜力，然而，近期研究引发了对这些模型倾向于生成表现出种族或性别偏见的输出的担忧。尽管训练数据可能是此类偏见的来源，但对文本数据中疾病与人口统计学关联的大规模探索仍然有限。方法：我们使用包含多样化网络来源（包括 Arxiv、Wikipedia 和 Common Crawl）的数据集开展了大规模文本分析。本研究分析了各种疾病与种族和性别标记一同被讨论的语境。鉴于 LLM 是在类似数据集上进行预训练的，该方法使我们能够考察 LLM 可能学习并内化的潜在偏见。我们将这些发现与实际人口统计学疾病患病率以及 GPT-4 的输出进行比较，以评估偏见表征的程度。结果：我们的发现表明，在线文本中，人口统计学术语与特定疾病概念存在不成比例的关联。性别术语与疾病概念显著相关，而种族术语的关联频率则低得多。我们发现，在特定种族和性别术语与所分析的18种疾病之间的关联中存在广泛差异。最突出的是，与人口比例相比，Black 种族提及总体上存在显著的过度表征。结论：我们的结果强调，需要对 LLM 预训练数据集中的偏见进行批判性审查和透明报告。本研究表明，需要制定缓解策略，以抵消 LLM 中有偏见训练数据的影响，尤其是在医疗保健等敏感领域。

英文摘要：

Background Advancements in Large Language Models (LLMs) hold transformative potential in healthcare, however, recent work has raised concern about the tendency of these models to produce outputs that display racial or gender biases. Although training data is a likely source of such biases, exploration of disease and demographic associations in text data at scale has been limited. Methods We conducted a large-scale textual analysis using a dataset comprising diverse web sources, including Arxiv, Wikipedia, and Common Crawl. The study analyzed the context in which various diseases are discussed alongside markers of race and gender. Given that LLMs are pre-trained on similar datasets, this approach allowed us to examine the potential biases that LLMs may learn and internalize. We compared these findings with actual demographic disease prevalence as well as GPT-4 outputs in order to evaluate the extent of bias representation. Results Our findings indicate that demographic terms are disproportionately associated with specific disease concepts in online texts. gender terms are prominently associated with disease concepts, while racial terms are much less frequently associated. We find widespread disparities in the associations of specific racial and gender terms with the 18 diseases analyzed. Most prominently, we see an overall significant overrepresentation of Black race mentions in comparison to population proportions. Conclusions Our results highlight the need for critical examination and transparent reporting of biases in LLM pretraining datasets. Our study suggests the need to develop mitigation strategies to counteract the influence of biased training data in LLMs, particularly in sensitive domains such as healthcare.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。