arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.09148cs.CLcs.IR
收藏

评估大语言模型在知名人物中的性别差异

Evaluating LLMs for Gender Disparities in Notable Persons

Robert H. Smith School of Business, University of Maryland(马里兰大学罗伯特·H·史密斯商学院);University of Antwerp(安特卫普大学);Stern School of Business, New York University(纽约大学斯特恩商学院)

Lauren Rhue, Sofie Goethals, Arun Sundararajan

AI总结:

本研究评估GPT系列大语言模型在事实检索中的性别差异,发现GPT-3.5存在明显性别偏见,GPT-4虽改进但未消除,尤其在拒绝回应时,并探讨了提示词性别关联与回应同质性的影响。

摘要:

本研究考察了使用大型语言模型(LLMs)检索事实信息的情况,解决了人们对它们倾向于产生事实上不正确的“幻觉”回应或完全拒绝回答提示的担忧。具体而言,它调查了LLMs在对事实性询问的回应中是否存在基于性别的偏见。本文采用多管齐下的方法评估GPT模型,通过评估在召回、幻觉和拒绝(弃权不执行)等多个维度上的公平性。我们的研究结果揭示了GPT-3.5生成的回应中存在明显的性别差异。虽然GPT-4的进步带来了性能上的改进,但并未完全消除这些性别差异,尤其是在回应被拒绝(弃权不执行)的情况下。该研究进一步通过考察提示中性别关联的影响和回应的同质性来探索这些差异的起源。

英文摘要:

This study examines the use of Large Language Models (LLMs) for retrieving factual information, addressing concerns over their propensity to produce factually incorrect "hallucinated" responses or to altogether decline to even answer prompt at all. Specifically, it investigates the presence of gender-based biases in LLMs' responses to factual inquiries. This paper takes a multi-pronged approach to evaluating GPT models by evaluating fairness across multiple dimensions of recall, hallucinations and declinations. Our findings reveal discernible gender disparities in the responses generated by GPT-3.5. While advancements in GPT-4 have led to improvements in performance, they have not fully eradicated these gender disparities, notably in instances where responses are declined. The study further explores the origins of these disparities by examining the influence of gender associations in prompts and the homogeneity in the responses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑