arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.04892cs.CL
收藏

偏见深藏:分配人设的大语言模型中的内隐推理偏见

Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs

Allen Institute for AI(艾伦人工智能研究所);Stanford University(斯坦福大学);Princeton University(普林斯顿大学)

Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, Tushar Khot

AI总结:

该研究首次系统探究人设分配对大语言模型推理能力的副作用,发现模型在公平表象下对不同社会人口群体存在深层内隐偏见,会引发性能大幅下降且难以规避,为当前流行的人设分配做法敲响警钟。

摘要:

近期研究展现了大语言模型(LLM)在回复中体现多样人设的能力,例如“你是尤达。请解释相对论”这类提示词就是典型例证。尽管这种能力实现了大语言模型的个性化,还能用于模拟人类行为,但其对大语言模型能力的影响仍不明确。为填补这一空白,我们首次针对人设分配对大语言模型基础推理任务能力的意外副作用开展了系统性研究。\n本研究覆盖24个推理数据集、4款大语言模型,以及跨越5个社会人口群体的19种不同人设(例如亚洲人)。实验揭示,大语言模型在公平的表象之下,对各类社会人口群体存在根深蒂固的偏见。当被明确问及(例如“黑人的数学能力更差吗?”)时,它们会公开拒绝刻板印象,但当被要求以某一人设回答问题时,就会表现出刻板化的错误预设。\n这些偏见可表现为回复中的弃权(不执行)行为,例如“作为一个黑人,我无法回答这个问题,因为它需要数学知识”,且通常会导致性能大幅下降。我们针对ChatGPT-3.5的实验表明,这种偏见普遍存在——80%的人设都表现出偏见;其影响显著——部分数据集的性能下降幅度超过70%;且对特定群体的危害尤其大——部分人设的性能在超过80%的数据集上出现统计意义上的显著下降。\n总体而言,4款大语言模型均不同程度地存在这种偏见,其中GPT-4-Turbo的偏见程度最低,但仍存在问题(42%的人设表现出偏见)。进一步分析显示,这些由人设引发的错误难以察觉,也难以避免。我们的研究结果敲响了警钟:当下日益流行的为大语言模型分配人设的做法,可能会暴露其深层偏见,并带来难以预见的有害副作用。

英文摘要:

Recent works have showcased the ability of LLMs to embody diverse personas in their responses, exemplified by prompts like 'You are Yoda. Explain the Theory of Relativity.' While this ability allows personalization of LLMs and enables human behavior simulation, its effect on LLMs' capabilities remains unclear. To fill this gap, we present the first extensive study of the unintended side-effects of persona assignment on the ability of LLMs to perform basic reasoning tasks. Our study covers 24 reasoning datasets, 4 LLMs, and 19 diverse personas (e.g. an Asian person) spanning 5 socio-demographic groups. Our experiments unveil that LLMs harbor deep rooted bias against various socio-demographics underneath a veneer of fairness. While they overtly reject stereotypes when explicitly asked ('Are Black people less skilled at mathematics?'), they manifest stereotypical and erroneous presumptions when asked to answer questions while adopting a persona. These can be observed as abstentions in responses, e.g., 'As a Black person, I can't answer this question as it requires math knowledge', and generally result in a substantial performance drop. Our experiments with ChatGPT-3.5 show that this bias is ubiquitous - 80% of our personas demonstrate bias; it is significant - some datasets show performance drops of 70%+; and can be especially harmful for certain groups - some personas suffer statistically significant drops on 80%+ of the datasets. Overall, all 4 LLMs exhibit this bias to varying extents, with GPT-4-Turbo showing the least but still a problematic amount of bias (evident in 42% of the personas). Further analysis shows that these persona-induced errors can be hard-to-discern and hard-to-avoid. Our findings serve as a cautionary tale that the practice of assigning personas to LLMs - a trend on the rise - can surface their deep-rooted biases and have unforeseeable and detrimental side-effects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑