超越表层线索：在多语言大语言模型（LLM）中解耦社会文化信号

中文摘要 AI 辅助

多语言大语言模型（LLM）的输出会随社会文化语境发生变化。然而，文化接地的证据可能具有误导性：身份标签可通过显性或隐性文本线索推断，而姓名与措辞可揭示源语言。将所有此类信号视为文化接地的证据，可能掩盖潜在偏见。本研究提出了经人工验证的多智能体审计方法，该方法区分三个问题：输出是否再现社会偏见、身份群体是否被不同表征、输出是否反映跨文化模式。研究分析了12个LLM在英语、法语、中文三种语言下的89253个输出，覆盖18种职业和三种任务条件。研究发现，偏见表征随语言和任务呈现系统性变化；移除直接身份线索会大幅降低英语和中文中的身份标签预测，但对法语影响小得多。在所有语言-体裁设置中，与源语言关联的文化语境获得最高平均相关性得分，自动评分与人工评分存在中等一致性；但在翻译后以及姓名掩码后，识别源语言的能力会大幅下降。若缺乏此类控制，多语言审计可能将表层线索误判为文化理解，进而得出关于跨文化变异与偏见的误导性结论。本审计提供了一种实用框架，用于将此类捷径与更具意义的跨文化模式解耦。

英文摘要

Multilingual LLM outputs can vary across sociocultural contexts. However, evidence of cultural grounding can be misleading: identity labels may be inferred from explicit or indirect textual cues, while names and wording can reveal the source language. Treating all these signals as evidence of cultural grounding may obscure potential biases. We present a human-validated, multi-agent audit that separates three questions: whether outputs reproduce social biases, whether identity groups are represented differently, and whether outputs reflect cross-cultural patterns. The study analyzes 89,253 outputs from 12 LLMs in English, French, and Chinese, spanning 18 occupations and three task conditions. We find that bias representation varies systematically across languages and tasks. Removing direct identity cues sharply reduces identity-label prediction in English and Chinese, but has a much smaller effect in French. Across all language-genre settings, the cultural context associated with the source language receives the highest average relevance score, with moderate agreement between automated and human ratings. However, the ability to identify the source language drops substantially after translation and again after masking names. Without these controls, multilingual audits may mistake surface cues for cultural understanding, leading to misleading conclusions about cross-cultural variation and bias. Our audit offers a practical framework for separating such shortcuts from more meaningful cross-cultural patterns.

发表机构

• University of Lausanne（洛桑大学）
• LMU Munich（慕尼黑大学）
• Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）

机构由 AI 辅助整理，请以论文原文为准。