通过性别和种族刻板印象评估大型语言模型
Evaluating Large Language Models through Gender and Racial Stereotypes
AI总结：
本研究建立评估框架，在专业背景下对比分析大型语言模型的性别和种族偏见，发现新模型中性别偏见大幅减少但种族偏见依然存在。
摘要：
语言模型开启了AI的新时代，在NLP社区和普通大众中获得了广泛关注。AI进行预测、生成以及在敏感决策场景中的应用能力，使得研究这些模型中可能存在且可能被夸大的潜在偏见变得更加重要。我们开展了一项质量比较研究，并建立了一个框架，在专业背景下基于两种偏见（性别和种族）的前提来评估语言模型。我们发现，与旧模型相比，新模型中的性别偏见已大幅减少，但种族偏见依然存在。
英文摘要：
Language Models have ushered a new age of AI gaining traction within the NLP community as well as amongst the general population. AI's ability to make predictions, generations and its applications in sensitive decision-making scenarios, makes it even more important to study these models for possible biases that may exist and that can be exaggerated. We conduct a quality comparative study and establish a framework to evaluate language models under the premise of two kinds of biases: gender and race, in a professional setting. We find out that while gender bias has reduced immensely in newer models, as compared to older ones, racial bias still exists.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
