文化差异保持：诊断LLM模拟调查群体中的扁平化与漫画化
Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populations
浏览论文内容
中文总结 AI 辅助
针对LLM模拟跨文化调查中传统保真度指标无法捕捉跨国差异的问题，提出文化差异保持（CDP）诊断方法，通过受控实验验证其单调性，并揭示DeepPersona提示虽保真度高却导致最强文化扁平化。
中文摘要 AI 辅助
大型语言模型（LLMs）越来越多地被用作合成调查受访者，以估计人群响应分布。在跨文化调查模拟中，评估不仅应关注国家内部的分布保真度，还应关注国家间差异是否得以保持。然而，现有的基于距离的度量（如Jensen-Shannon散度（JSD））并不能直接捕捉这种跨国差异。为解决这一局限，我们引入了文化差异保持（CDP），这是一种基于一次性人工校准的轻参考诊断方法。CDP将跨国差异的减少识别为文化扁平化，将差异的增加识别为文化漫画化。为评估CDP，我们在四个LLM骨干模型、三种基于角色的提示方法和两个调查领域（世界价值观调查（WVS）和大五人格测试）上进行了实验。结果揭示了传统保真度度量与CDP之间的系统性差异。受控实验表明，随着跨国差异被减弱或放大，CDP呈单调变化，而JSD的相应变化相对较小。在我们对真实LLM生成的审计中，DeepPersona启发式提示经常被传统保真度度量所青睐，但在每个模型-领域块中表现出最强的扁平化。因此，CDP通过直接量化跨国差异的减弱或放大，补充了保真度度量。
英文摘要
Large language models (LLMs) are increasingly used as synthetic survey respondents to estimate population response distributions. In cross-cultural survey simulation, evaluations should assess not only distributional fidelity within countries but also whether differences across countries are preserved. However, existing distance-based metrics such as Jensen--Shannon divergence (JSD) do not directly capture such cross-country differences. To address this limitation, we introduce Cultural Divergence Preservation (CDP), a reference-light diagnostic based on a one-time human calibration. CDP identifies reduced cross-country divergence as cultural flattening and increased divergence as cultural caricature. To evaluate CDP, we conduct experiments across four LLM backbones, three persona-based prompting methods, and two survey domains, the World Values Survey (WVS) and the Big Five Personality Test. The results reveal a systematic discrepancy between conventional fidelity metrics and CDP. Controlled experiments show that CDP changes monotonically as cross-country divergence is attenuated or amplified, while the corresponding changes in JSD remain relatively small. In our audit of real LLM generations, DeepPersona-Inspired prompting is frequently favored by conventional fidelity metrics but exhibits the strongest flattening in every model--domain block. CDP thus complements fidelity metrics by directly quantifying the attenuation or amplification of cross-country divergence.
发表机构
- Yonsei University（延世大学）
- NAVER
- Seoul National University Hospital（首尔大学医院）
机构由 AI 辅助整理，请以论文原文为准。
