文化真实性：比较大语言模型的文化表征与本地人类期望
Cultural Authenticity: Comparing LLM Cultural Representations to Native Human Expectations
AI总结：
本文提出人类中心框架评估LLM生成内容与本地期望的一致性，发现部分模型存在以西方为中心的校准，文化距离越远一致性越低，并识别出系统性误差。
摘要：
大语言模型（LLM）输出中的文化表征主要通过文化多样性与事实准确性来评估。然而，评估文化一致性仍存在关键缺口：生成内容是否符合本地人口对自身文化要素的感知和优先级。本文引入人类中心框架评估LLM生成内容与本地期望的一致性。首先，基于九个国家开放性调查收集的数据，建立人类衍生的基准重要向量，称为文化重要向量。接着，通过语法多样化的提示集计算LLM的文化表征向量，并应用于三个前沿LLM（Gemini 2.5 Pro、GPT-4o和Claude 3.5 Haiku）。研究发现，部分模型存在以西方为中心的校准，文化距离越远一致性越低。此外，所有模型均表现出高度相关的系统性误差（ρ>0.97），在某些文化标记上过度索引，而忽视了用户深层次的社会和价值优先级。本文方法超越简单多样性指标，评估AI生成内容在真实捕捉全球文化细微层次方面的一致性。
英文摘要：
Cultural representation in Large Language Model (LLM) outputs has primarily been evaluated through the proxies of cultural diversity and factual accuracy. However, a crucial gap remains in assessing cultural alignment: the degree to which generated content mirrors how native populations perceive and prioritize their own cultural facets. In this paper, we introduce a human-centered framework to evaluate the alignment of LLM generations with local expectations. First, we establish a human-derived ground-truth baseline of importance vectors, called Cultural Importance Vectors based on an induced set of culturally significant facets from open-ended survey responses collected across nine countries. Next, we introduce a method to compute model-derived Cultural Representation Vectors of an LLM based on a syntactically diversified prompt-set and apply it to three frontier LLMs (Gemini 2.5 Pro, GPT-4o, and Claude 3.5 Haiku). Our investigation of the alignment between the human-derived Cultural Importance and model-derived Cultural Representations reveals a Western-centric calibration for some of the models where alignment decreases as a country's cultural distance from the US increases. Furthermore, we identify highly correlated, systemic error signatures ($ρ> 0.97$) across all models, which over-index on some cultural markers while neglecting the deep-seated social and value-based priorities of users. Our approach moves beyond simple diversity metrics toward evaluating the fidelity of AI-generated content in authentically capturing the nuanced hierarchies of global cultures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
