探究大语言模型中更隐蔽的偏见：生成式模型中的年龄歧视、容貌偏见、机构偏见与国籍偏见
Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models
AI总结：
该研究聚焦LLM中较少被关注的年龄、容貌等维度的隐蔽偏见，构建模板生成的双向句子补全数据集，在4款前沿LLM上测试偏见关联，为通用偏见评估提供可扩展基准。
摘要：
大语言模型（LLMs）的能力日益强大，被广泛用于辅助用户完成各类任务。这种应用存在将LLM的偏见引入招聘、人员绩效评估、刑事量刑等重大决策的风险。自然语言处理系统中基于性别和种族的偏见已被广泛研究，尤其是针对特定刻板印象（例如亚洲人擅长数学）的研究。\n本文中，我们探究了研究较少但仍具重要影响的维度上的偏见，比如年龄和容貌，衡量LLM在社会群体与无关的正负属性之间做出的更隐蔽的关联决策。我们探究LLM是否对特定社会群体持有广泛的正面或负面情绪偏见，类似于实验心理学中在人类身上发现的“美即好”偏见。\n我们构建了一个由模板生成的句子补全任务数据集，要求模型选择最合适的属性，来补全关于被描述为特定社会群体成员的人物的评价性陈述。我们还反转了补全任务，即根据属性选择对应的社会群体。我们报告了在4款前沿LLM上发现的关联结果。该数据集可作为基准，用于评估更广泛偏见方面的研究进展，且这种模板生成技术可用于扩展该基准，只需极少的额外人工标注。
英文摘要：
LLMs are increasingly powerful and widely used to assist users in a variety of tasks. This use risks the introduction of LLM biases to consequential decisions such as job hiring, human performance evaluation, and criminal sentencing. Bias in NLP systems along the lines of gender and ethnicity has been widely studied, especially for specific stereotypes (e.g., Asians are good at math). In this paper, we investigate bias along less-studied but still consequential, dimensions, such as age and beauty, measuring subtler correlated decisions that LLMs make between social groups and unrelated positive and negative attributes. We ask whether LLMs hold wide-reaching biases of positive or negative sentiment for specific social groups similar to the "what is beautiful is good" bias found in people in experimental psychology. We introduce a template-generated dataset of sentence completion tasks that asks the model to select the most appropriate attribute to complete an evaluative statement about a person described as a member of a specific social group. We also reverse the completion task to select the social group based on an attribute. We report the correlations that we find for 4 cutting-edge LLMs. This dataset can be used as a benchmark to evaluate progress in more generalized biases and the templating technique can be used to expand the benchmark with minimal additional human annotation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
