在交叉性语境中评估语言模型的偏见态度关联
Evaluating Biased Attitude Associations of Language Models in an Intersectional Context
AI总结：
本文提出基于概念投影的效价子空间方法，在交叉性语境中量化英语语言模型对12类社会群体的偏见态度关联，发现模型对性别认同、社会阶层和性取向偏见最强，且性能越好的模型偏见越明显。
摘要：
语言模型在大规模语料库上训练，这些语料库嵌入了心理学文献中记载的内隐偏见。社会群体的效价关联（愉悦性/不愉悦性）决定了社会认知中对群体和概念的偏见态度。基于这一既有文献，我们使用提供交叉性语境的句子模板，量化英语语言模型中社会群体被赋予效价的方式。我们研究与年龄、教育、性别、身高、智力、读写能力、种族、宗教、生理性别、性取向、社会阶层和体重相关的偏见。我们提出了一种概念投影方法，通过语言模型的语境化词嵌入来捕获效价子空间。将基于投影的方法适配到用于量化偏见的嵌入关联测试中，我们发现语言模型对语言中的性别认同、社会阶层和性取向信号表现出最强烈的偏见态度。我们发现，我们所研究的规模最大且性能更好的模型也更具偏见，因为它有效地捕获了嵌入在社会文化数据中的偏见。我们通过在一个内在效价评估任务上取得更优表现来验证该偏见评估方法。该方法使我们能够测量复杂的交叉性偏见，因为这些偏见已知会体现在语言模型的输出和应用中，从而延续历史偏见。此外，我们的方法有助于设计正义，因为它研究了在语言中代表性不足的群体（如跨性别者和同性恋者）的关联。
英文摘要：
Language models are trained on large-scale corpora that embed implicit biases documented in psychology. Valence associations (pleasantness/unpleasantness) of social groups determine the biased attitudes towards groups and concepts in social cognition. Building on this established literature, we quantify how social groups are valenced in English language models using a sentence template that provides an intersectional context. We study biases related to age, education, gender, height, intelligence, literacy, race, religion, sex, sexual orientation, social class, and weight. We present a concept projection approach to capture the valence subspace through contextualized word embeddings of language models. Adapting the projection-based approach to embedding association tests that quantify bias, we find that language models exhibit the most biased attitudes against gender identity, social class, and sexual orientation signals in language. We find that the largest and better-performing model that we study is also more biased as it effectively captures bias embedded in sociocultural data. We validate the bias evaluation method by overperforming on an intrinsic valence evaluation task. The approach enables us to measure complex intersectional biases as they are known to manifest in the outputs and applications of language models that perpetuate historical biases. Moreover, our approach contributes to design justice as it studies the associations of groups underrepresented in language such as transgender and homosexual individuals.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
