arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.10484cs.CYcs.AI

AI伦理困境中决策的偏见:ChatGPT与Claude的比较研究

Bias in Decision-Making for AI's Ethical Dilemmas: A Comparative Study of ChatGPT and Claude

Wentao Xu, Yile Yan, Yuqi Zhu

AI总结:

本研究系统评估了九个LLMs在涉及受保护属性的伦理困境中的决策偏见,发现开源与闭源模型在不同困境情境下表现出不同的伦理偏好与敏感性,交叉属性输入揭示了更深层的偏见。

摘要:

大型语言模型(LLMs)的近期进展使其能够在各种任务中生成类人响应,这引发了关于其伦理决策能力和潜在偏见的疑问。本研究系统评估了九个流行的LLMs(包括开源和闭源)如何应对涉及受保护属性的伦理困境。在跨越四种困境场景(保护性场景与有害场景)中单一和交叉属性组合的50,400次试验中,我们评估了模型的伦理偏好、敏感性、稳定性和聚类模式。结果显示,所有模型在受保护属性上均存在显著偏见,且偏好因模型类型和困境情境而异。值得注意的是,开源LLMs对边缘化群体表现出更强的偏好,且在有害场景中敏感性更高;而闭源模型在保护性情境中更具选择性,并倾向于偏好主流群体。我们还发现,伦理行为因困境类型而异:LLMs在保护性场景中保持一致的模式,但在有害场景中则做出更多样化、认知需求更高的决策。此外,模型在交叉条件下的伦理倾向比在单一属性设置中更为显著,表明复杂输入揭示了更深层的偏见。这些发现强调了对LLMs伦理行为进行多维、情境感知评估的必要性,并为理解和解决LLM决策中的公平性提供了系统评估方法。

英文摘要:

Recent advances in Large Language Models (LLMs) have enabled human-like responses across various tasks, raising questions about their ethical decision-making capabilities and potential biases. This study systematically evaluates how nine popular LLMs (both open-source and closed-source) respond to ethical dilemmas involving protected attributes. Across 50,400 trials spanning single and intersectional attribute combinations in four dilemma scenarios (protective vs. harmful), we assess models' ethical preferences, sensitivity, stability, and clustering patterns. Results reveal significant biases in protected attributes in all models, with differing preferences depending on model type and dilemma context. Notably, open-source LLMs show stronger preferences for marginalized groups and greater sensitivity in harmful scenarios, while closed-source models are more selective in protective situations and tend to favor mainstream groups. We also find that ethical behavior varies across dilemma types: LLMs maintain consistent patterns in protective scenarios but respond with more diverse and cognitively demanding decisions in harmful ones. Furthermore, models display more pronounced ethical tendencies under intersectional conditions than in single-attribute settings, suggesting that complex inputs reveal deeper biases. These findings highlight the need for multi-dimensional, context-aware evaluation of LLMs' ethical behavior and offer a systematic evaluation and approach to understanding and addressing fairness in LLM decision-making.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑