大语言模型中的文化偏差:通过道德问卷评估AI智能体
Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires
AI总结:
该研究通过在19种文化语境下应用道德基础问卷,发现大语言模型会系统性同质化道德多样性,模型规模增大也无法持续提升文化表征保真度,对LLMs用作合成人口的做法提出质疑,呼吁优化对齐目标与评估指标以适配多元人类价值观。
摘要:
AI系统是真正代表人类价值观,还是仅仅对其进行平均化处理?我们的研究揭示了一个令人担忧的现实:大语言模型(LLMs)尽管具备语言能力,却无法代表多元的文化道德框架。我们通过在19种文化语境中应用道德基础问卷(Moral Foundations Questionnaire),揭示了AI生成的道德直觉与人类道德直觉之间存在显著差距。将多个前沿LLMs的输出与人类基准数据进行对比后,我们发现这些模型系统性地同质化了道德多样性。令人意外的是,模型规模的增大并不能持续提升文化表征的保真度。我们的研究结果对社会科学研究中日益增多的将LLMs用作合成人口的做法提出了质疑,并凸显了当前AI对齐方法的一个根本局限。若仅依赖提示工程、缺乏数据驱动的对齐,这些系统无法捕捉细致的、具有文化特异性的道德直觉。我们的结果呼吁制定更贴合实际的对齐目标与评估指标,以确保AI系统代表多元人类价值观,而非将道德格局扁平化。
英文摘要:
Are AI systems truly representing human values, or merely averaging across them? Our study suggests a concerning reality: Large Language Models (LLMs) fail to represent diverse cultural moral frameworks despite their linguistic capabilities. We expose significant gaps between AI-generated and human moral intuitions by applying the Moral Foundations Questionnaire across 19 cultural contexts. Comparing multiple state-of-the-art LLMs' origins against human baseline data, we find these models systematically homogenize moral diversity. Surprisingly, increased model size doesn't consistently improve cultural representation fidelity. Our findings challenge the growing use of LLMs as synthetic populations in social science research and highlight a fundamental limitation in current AI alignment approaches. Without data-driven alignment beyond prompting, these systems cannot capture the nuanced, culturally-specific moral intuitions. Our results call for more grounded alignment objectives and evaluation metrics to ensure AI systems represent diverse human values rather than flattening the moral landscape.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。