大型语言模型中的表征偏差有多深？种姓与宗教的案例
How Deep Is Representational Bias in LLMs? The Cases of Caste and Religion
AI总结：
本研究系统审计GPT-4 Turbo，发现其在宗教和种姓表征上存在深层且粘性的偏差，过度代表文化主导群体，提示干预效果有限，表明仅多样化训练数据不足以纠正偏差。
摘要：
大型语言模型（LLM）中的表征偏差主要通过单次交互测量，且聚焦于种族和性别等以全球北方为中心的身份维度。我们扩展了该研究，对GPT-4 Turbo进行系统性审计，以揭示表征偏差被编码的深度及其如何延伸至较少探索的身份维度。我们提示GPT-4 Turbo生成超过7200个关于印度重大生活事件（如婚礼）的故事，使用旨在不同程度鼓励多样性的提示。将输出中宗教和种姓表征的多样性与实际人口分布（依据人口普查数据）进行比较，我们量化了LLM中宗教和种姓表征偏差的存在及其“粘性”。我们发现，尽管提示旨在鼓励表征多样性，GPT-4的响应始终过度代表文化主导群体，远超其统计代表性。我们的发现还表明，LLM中的表征偏差具有赢家通吃的特性，比其训练数据中可能的分布偏差更严重，且重复的基于提示的推动在消除这些偏差方面效果有限且不一致。这些结果表明，仅多样化训练数据可能不足以纠正LLM偏差，凸显了在模型开发中进行更根本性变革的必要性。数据集和代码手册：https://github.com/agrimaseth/How-Deep-Is-Representational-Bias-in-LLMs
英文摘要：
Representational bias in large language models (LLMs) has predominantly been measured through single-response interactions and has focused on Global North-centric identities like race and gender. We expand on that research by conducting a systematic audit of GPT-4 Turbo to reveal how deeply encoded representational biases are and how they extend to less-explored dimensions of identity. We prompt GPT-4 Turbo to generate over 7,200 stories about significant life events (such as weddings) in India, using prompts designed to encourage diversity to varying extents. Comparing the diversity of religious and caste representation in the outputs against the actual population distribution in India as recorded in census data, we quantify the presence and "stickiness" of representational bias in the LLM for religion and caste. We find that GPT-4 responses consistently overrepresent culturally dominant groups far beyond their statistical representation, despite prompts intended to encourage representational diversity. Our findings also suggest that representational bias in LLMs has a winner-take-all quality that is more biased than the likely distribution bias in their training data, and repeated prompt-based nudges have limited and inconsistent efficacy in dislodging these biases. These results suggest that diversifying training data alone may not be sufficient to correct LLM bias, highlighting the need for more fundamental changes in model development. Dataset and Codebook: https://github.com/agrimaseth/How-Deep-Is-Representational-Bias-in-LLMs
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
