通过合成人物画像模拟评估大语言模型的文化适应性
Evaluating Cultural Adaptability of a Large Language Model via Simulation of Synthetic Personas
AI总结：
本研究通过让GPT-3.5模拟15国7286名参与者对说服性新闻的反应，发现指定居住国可提升模型文化适配性，而母语提示反而会降低模拟保真度。
摘要：
大语言模型（LLMs）在多元文化环境中的成功，取决于其理解用户多样文化背景的能力。我们通过让LLM在问卷式心理学实验范畴内模拟代表不同国籍的人类画像，来衡量这一能力。具体而言，我们使用GPT-3.5复现来自15个国家的7286名参与者对说服性新闻文章的反应，并将结果与具有相同人口统计学特征的真实参与者数据集进行比较。我们的分析表明，指定一个人的居住国能够提升GPT-3.5的回答与真实参与者回答的一致性。相比之下，使用母语提示会带来偏差，显著降低整体一致性，部分语言尤其会损害性能。这些发现表明，虽然直接的国籍信息能增强模型的文化适应性，但母语线索并不能可靠地提升模拟保真度，反而可能削弱模型的有效性。
英文摘要：
The success of Large Language Models (LLMs) in multicultural environments hinges on their ability to understand users' diverse cultural backgrounds. We measure this capability by having an LLM simulate human profiles representing various nationalities within the scope of a questionnaire-style psychological experiment. Specifically, we employ GPT-3.5 to reproduce reactions to persuasive news articles of 7,286 participants from 15 countries; comparing the results with a dataset of real participants sharing the same demographic traits. Our analysis shows that specifying a person's country of residence improves GPT-3.5's alignment with their responses. In contrast, using native language prompting introduces shifts that significantly reduce overall alignment, with some languages particularly impairing performance. These findings suggest that while direct nationality information enhances the model's cultural adaptability, native language cues do not reliably improve simulation fidelity and can detract from the model's effectiveness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
