探究大语言模型生成故事中的性别偏见：基于心理刻板印象的视角
Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypes
AI总结：
本研究通过心理刻板印象在叙事生成中探究大语言模型的性别偏见，提出StereoBias-Stories数据集，发现属性条件化可缓解偏见，组合属性加剧或缓解偏见，且模型偏见与心理真实情况一致。
摘要：
随着大语言模型（LLMs）在不同应用中的日益普及，人们对其在各类任务中可能放大性别偏见的担忧也在上升。以往的研究通常使用显性性别线索作为反事实来探测性别偏见，或在句子补全和简短问答任务中研究该问题。这些形式可能忽略了嵌入在较长内容生成行为中的更隐性的偏见形式。在本工作中，我们利用心理学中研究的性别刻板印象（如攻击性或八卦）在开放式叙事生成任务中探究大语言模型的性别偏见。我们引入了一个名为StereoBias-Stories的新数据集，其中包含短篇故事，这些故事要么是无条件的，要么是以来自25种心理刻板印象的（一个、两个或六个）随机属性和三个与任务相关的故事结局为条件的。我们分析了整体故事中的性别贡献如何响应这些属性而变化，并提出了三个关键发现：（1）虽然模型在无条件提示中平均高度偏向男性，但以与性别刻板印象无关的属性为条件可以缓解这种偏见。（2）将多个与同一性别刻板印象相关的属性组合会加剧模型行为，其中男性属性放大偏见，而女性属性则缓解偏见。（3）模型偏见与用于分类的心理真实情况一致，且一致性强度随模型规模增大而增强。这些见解共同凸显了基于心理学的大语言模型评估的重要性。
英文摘要：
As Large Language Models (LLMs) are increasingly used across different applications, concerns about their potential to amplify gender biases in various tasks are rising. Prior research has often probed gender bias using explicit gender cues as counterfactual, or studied them in sentence completion and short question answering tasks. These formats might overlook more implicit forms of bias embedded in generative behavior of longer content. In this work, we investigate gender bias in LLMs using gender stereotypes studied in psychology (e.g., aggressiveness or gossiping) in an open-ended task of narrative generation. We introduce a novel dataset called StereoBias-Stories containing short stories either unconditioned or conditioned on (one, two, or six) random attributes from 25 psychological stereotypes and three task-related story endings. We analyze how the gender contribution in the overall story changes in response to these attributes and present three key findings: (1) While models, on average, are highly biased towards male in unconditioned prompts, conditioning on attributes independent from gender stereotypes mitigates this bias. (2) Combining multiple attributes associated with the same gender stereotype intensifies model behavior, with male ones amplifying bias and female ones alleviating it. (3) Model biases align with psychological ground-truth used for categorization, and alignment strength increases with model size. Together, these insights highlight the importance of psychology-grounded evaluation of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
