位置未找到：揭示多语言大语言模型中的隐含本地和全球偏见
Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs
AI总结：
本文通过LocQA测试集研究多语言模型的本地和跨语言偏见，发现模型对美国本地信息有全球偏见，并在指令微调模型中更显著，同时在同一种语言内优先考虑人口多的地区。
摘要：
多语言大语言模型（LLMs）已最小化了语言间的流畅性差距。然而，这一进步使模型面临偏见行为的风险，因为知识和规范可能在语言间传播。本文旨在通过模型回答模糊地域问题的能力来量化模型的跨语言和内语言偏见。为此，我们提出了LocQA，包含12种语言中的2156个问题，涉及各种依赖地域的事实，如法律、日期和测量。这些问题不包含关于其相关地域的指示，除了查询语言本身。LLMs对LocQA模糊地域问题的回应因此揭示了模型的隐含先验。我们使用LocQA评估了32个模型，检测到两种类型的结构性偏见。跨语言方面，我们展示了对美国地域相关信息的全球偏见，即使在非英语语言中提问时也是如此。此外，我们发现这种全球偏见在经过指令微调的模型中比基线模型更显著。在内语言方面，我们发现当同一语言涉及多个地域时，模型表现为人口概率引擎，优先考虑人口较多的地域。总体而言，LocQA的见解可能有助于塑造LLMs期望的本地行为，并量化不同训练阶段对不同偏见类型的影响。
英文摘要：
Multilingual large language models (LLMs) have minimized the fluency gap between languages. This advancement, however, exposes models to the risk of biased behavior, as knowledge and norms may propagate across languages. In this work, we aim to quantify models' inter- and intra-lingual biases, via their ability to answer locale-ambiguous questions. To this end, we present LocQA, a test set containing 2,156 questions in 12 languages, referring to various locale-dependent facts such as laws, dates, and measurements. The questions do not contain indications of the locales they relate to, other than the querying language itself. LLMs' responses to LocQA locale-ambiguous questions thus reveal models' implicit priors. We used LocQA to evaluate 32 models, and detected two types of structural biases. Inter-lingually, we show a global bias towards answers relevant to the US-locale, even when models are asked in languages other than English. Moreover, we discovered that this global bias is exacerbated in models that underwent instruction tuning, compared to their base counterparts. Intra-lingually, we show that when multiple locales are relevant for the same language, models act as demographic probability engines, prioritizing locales with larger populations. Taken together, insights from LocQA may help in shaping LLMs' desired local behavior, and in quantifying the impact of various training phases on different kinds of biases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
