为什么所有大语言模型（LLMs）都痴迷于日本文化？关于大语言模型的隐藏文化与区域偏见
Why are all LLMs Obsessed with Japanese Culture? On the Hidden Cultural and Regional Biases of LLMs
AI总结：
本研究构建了涵盖24种语言的文化相关开放问题（CROQ）数据集，发现LLMs在文化问题回答中存在偏向日本等国家的区域偏见，且该偏见在监督微调后出现。
摘要：
大语言模型（LLMs）在文化覆盖与文化能力方面存在局限性，部分情况下会表现出特定的文化偏见。尽管已有研究考察了LLMs的文化能力，但尚未有研究专门调查它们在通用文化相关问题中的区域偏好。本研究基于文化相关开放问题（CROQ）的综合分类法构建了一个新数据集，该数据集的问题涵盖24种语言。我们通过提示LLMs回答CROQ中的问题并提供示例位置来对其进行评估。结果显示，与以往文化偏见研究的结论相反，LLMs在回答中表现出明显偏向日本等国家的倾向。此外，我们的结果表明，当用英语或其他高资源语言进行提示时，LLMs倾向于提供更多样化的输出；而低资源语言则更倾向于回答突出输入语言为官方语言的国家的问题。最后，我们还研究了LLMs训练过程中文化偏见出现的阶段，结果显示，首个明显迹象出现在监督微调后，而非预训练阶段。数据集可通过此链接获取：https://this URL
英文摘要：
LLMs have limitations when it comes to cultural coverage and competence, and in some cases, show specific cultural biases. Although prior studies have examined the cultural capabilities of LLMs, none have specifically investigated their regional preferences in generic culture-related questions. In this work, we propose a new dataset based on a comprehensive taxonomy of Culture-Related Open Questions (CROQ), with questions available in 24 languages. We evaluate LLMs by prompting them to answer questions from CROQ and provide a sample location. The results show that, contrary to previous cultural bias work, LLMs show a clear tendency towards countries such as Japan in their answers. Moreover, our results show that when prompting in languages such as English or other high-resource ones, LLMs tend to provide more diverse outputs. Low-resource languages, on the other hand, show more inclinations towards answering questions highlighting countries for which the input language is an official language. Finally, we also investigate at which point of LLM training this cultural bias emerges, with our results suggesting that the first clear signs appear after supervised fine-tuning, and not during pre-training. Dataset available at https://huggingface.co/datasets/HiTZ/CROQ
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
