当LLMs想象人们：一种以人为中心的人格脑暴审计方法，用于创意应用中的偏见和公平性
When LLMs Imagine People: A Human-Centered Persona Brainstorm Audit for Bias and Fairness in Creative Applications
AI总结：
本文提出了一种人格脑暴审计方法，用于评估LLM在创意应用中的偏见和公平性，通过量化偏见并分析多个交集身份和社会角色，揭示模型生成中潜在的不公平现象。
摘要：
大型语言模型（LLMs）在创意工作流程中使用时，可能会强化刻板印象并加剧不平等，因此公平性审计至关重要。现有方法依赖于受限任务和固定基准，无法对开放性创意输出进行审查。我们引入了人格脑暴审计（PBA），这是一种可扩展且易于扩展的偏见检测审计方法，用于在开放性人格生成中多个交集的身份和社会角色。PBA使用自由度意识的标准化Cramér's V来量化偏见，产生可解释的严重性标签，使模型和维度之间的公平比较成为可能。将PBA应用于12个LLMs（120,000个人格，16个偏见维度），我们发现偏见在模型生成中呈非线性变化：较大的和较新的模型并不总是更公平，而且最初减少的偏见可能在后续版本中再次出现。交叉分析揭示了单轴指标掩盖的不平等，其中单独看起来公平的维度在组合中可能表现出高偏见。稳健性分析显示PBA在不同样本量、角色扮演提示和去偏见提示下保持稳定，确立了其在LLM公平性审计中的可靠性。
英文摘要：
Large Language Models (LLMs) used in creative workflows can reinforce stereotypes and perpetuate inequities, making fairness auditing essential. Existing methods rely on constrained tasks and fixed benchmarks, leaving open-ended creative outputs unexamined. We introduce the Persona Brainstorm Audit (PBA), a scalable and easy to extend auditing method for bias detection across multiple intersecting identity and social roles in open-ended persona generation. PBA quantifies bias using degree-of-freedom-aware normalized Cramér's V, producing interpretable severity labels that enable fair comparison across models and dimensions. Applying PBA to 12 LLMs (120,000 personas, 16 bias dimensions), we find that bias evolves nonlinearly across model generations: larger and newer models are not consistently fairer, and biases that initially decrease can resurface in later releases. Intersectional analysis reveals disparities hidden by single-axis metrics, where dimensions appearing fair individually can exhibit high bias in combination. Robustness analyses show PBA remains stable under varying sample sizes, role-playing prompts, and debiasing prompts, establishing its reliability for fairness auditing in LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
