LLMs的歧视：决策与摘要任务中的跨语言偏见评估与缓解
Discrimination by LLMs: Cross-lingual Bias Assessment and Mitigation in Decision-Making and Summarisation
AI总结：
本研究评估了GPT-3.5和GPT-4o在决策与摘要任务中跨语言（英语与荷兰语）的背景、性别及年龄偏见，发现决策任务偏见显著而摘要任务较小，并验证了提示指令策略能有效缓解但无法完全消除偏见。
摘要：
大型语言模型（LLMs）快速融入各个领域，引发了对社会不平等和信息偏见的担忧。本研究考察了LLMs中与背景、性别和年龄相关的偏见，重点关注其对决策和摘要任务的影响。此外，研究还考察了这些偏见的跨语言传播情况，并评估了基于提示指令的缓解策略的有效性。使用由Tamkin等人（2023）提出的数据集的改编版本并翻译成荷兰语，我们为决策任务创建了151,200个独特的提示，为摘要任务创建了176,400个提示。在GPT-3.5和GPT-4o上测试了各种人口统计变量、指令、显著度水平和语言。我们的分析显示，两个模型在决策过程中都存在显著偏见，偏好女性、较年轻的年龄以及某些背景，如非裔美国人背景。相比之下，摘要任务几乎没有显示出偏见的证据，尽管GPT-3.5在英语中出现了显著的年龄相关差异。跨语言分析表明，英语和荷兰语之间的偏见模式总体相似，但在特定人口统计类别中观察到了显著差异。新提出的缓解指令虽然无法完全消除偏见，但展示了减少偏见的潜力。最有效的指令在缩小最有利和最不利人群之间的差距方面实现了27%的平均减少量。值得注意的是，与GPT-3.5相反，GPT-4o在所有英语提示中都显示出偏见减少，这表明较新模型在基于提示的缓解方面具有特定潜力。本研究强调了谨慎采用LLMs和特定情境偏见测试的重要性，突出了持续开发有效缓解策略以确保负责任地部署AI的需求。
英文摘要：
The rapid integration of Large Language Models (LLMs) into various domains raises concerns about societal inequalities and information bias. This study examines biases in LLMs related to background, gender, and age, with a focus on their impact on decision-making and summarization tasks. Additionally, the research examines the cross-lingual propagation of these biases and evaluates the effectiveness of prompt-instructed mitigation strategies. Using an adapted version of the dataset by Tamkin et al. (2023) translated into Dutch, we created 151,200 unique prompts for the decision task and 176,400 for the summarisation task. Various demographic variables, instructions, salience levels, and languages were tested on GPT-3.5 and GPT-4o. Our analysis revealed that both models were significantly biased during decision-making, favouring female gender, younger ages, and certain backgrounds such as the African-American background. In contrast, the summarisation task showed minimal evidence of bias, though significant age-related differences emerged for GPT-3.5 in English. Cross-lingual analysis showed that bias patterns were broadly similar between English and Dutch, though notable differences were observed across specific demographic categories. The newly proposed mitigation instructions, while unable to eliminate biases completely, demonstrated potential in reducing them. The most effective instruction achieved a 27\% mean reduction in the gap between the most and least favorable demographics. Notably, contrary to GPT-3.5, GPT-4o displayed reduced biases for all prompts in English, indicating the specific potential for prompt-based mitigation within newer models. This research underscores the importance of cautious adoption of LLMs and context-specific bias testing, highlighting the need for continued development of effective mitigation strategies to ensure responsible deployment of AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
