行动胜于言辞：智能体决策揭示语言模型中的隐性偏见
Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models
AI总结：
本文提出一种通过评估智能体决策差异来系统揭示LLM在模拟人类行为时隐性偏见的技术，发现更先进的模型隐性偏见更大且对现实差异存在放大效应。
摘要：
尽管在公平性和对齐方面的进展有助于缓解大型语言模型（LLM）在显式提示下表现出的显性偏见，但我们假设这些模型在模拟人类行为时可能仍会表现出隐性偏见。为了检验这一假设，我们提出了一种技术，通过评估具有 LLM 生成的、具备社会人口学特征的人设的智能体之间的决策差异，系统地揭示广泛社会人口学类别中的此类偏见。使用我们的技术，我们在三个社会人口学群体和四个决策场景中测试了六个 LLM。结果显示，最先进的 LLM 在几乎所有模拟中都表现出显著的社会人口学差异，更先进的模型尽管减少了显性偏见，却表现出更大的隐性偏见。此外，将我们的发现与实证研究中报告的现实世界差异进行比较时，我们发现所揭示的偏见在方向上是一致的，但被显著放大了。这种方向上的一致性突显了我们的技术在揭示 LLM 系统性偏见而非随机变化方面的效用；此外，隐性偏见的存在和放大强调了解决这些偏见需要新的策略。
英文摘要：
While advances in fairness and alignment have helped mitigate overt biases exhibited by large language models (LLMs) when explicitly prompted, we hypothesize that these models may still exhibit implicit biases when simulating human behavior. To test this hypothesis, we propose a technique to systematically uncover such biases across a broad range of sociodemographic categories by assessing decision-making disparities among agents with LLM-generated, sociodemographically-informed personas. Using our technique, we tested six LLMs across three sociodemographic groups and four decision-making scenarios. Our results show that state-of-the-art LLMs exhibit significant sociodemographic disparities in nearly all simulations, with more advanced models exhibiting greater implicit biases despite reducing explicit biases. Furthermore, when comparing our findings to real-world disparities reported in empirical studies, we find that the biases we uncovered are directionally aligned but markedly amplified. This directional alignment highlights the utility of our technique in uncovering systematic biases in LLMs rather than random variations; moreover, the presence and amplification of implicit biases emphasizes the need for novel strategies to address these biases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
