结合训练方法论探索大语言模型的跨文化价值观
Exploring Large Language Models on Cross-Cultural Values in Connection with Training Methodology
AI总结：
本研究探究开源大语言模型的跨文化价值观判断能力及其与训练方法论的关系，发现模型存在西方文化偏向，多语言语料、增大模型规模或合成数据可优化相关能力，为LLM设计提供洞见。
摘要：
大语言模型（LLMs）与人类交互密切，因此需要深入理解人类社会的文化价值观。本文探究了开源LLMs如何对不同国家的各类文化价值观做出判断，以及该能力与模型规模、训练语料、对齐方式等训练方法论的关联。分析表明，LLMs对社会文化规范的判断与人类相近，但对社会制度与发展的判断契合度较低。此外，LLMs的文化价值观判断倾向于偏向西方文化，通过多语言语料训练可改善这一偏差。研究还发现，增大模型规模有助于提升对社会价值观的理解，而小模型可通过使用合成数据增强该能力。本分析为LLMs的设计方法论与其文化价值观理解的关联提供了宝贵洞见。
英文摘要：
Large language models (LLMs) closely interact with humans, and thus need an intimate understanding of the cultural values of human society. In this paper, we explore how open-source LLMs make judgments on diverse categories of cultural values across countries, and its relation to training methodology such as model sizes, training corpus, alignment, etc. Our analysis shows that LLMs can judge socio-cultural norms similar to humans but less so on social systems and progress. In addition, LLMs tend to judge cultural values biased toward Western culture, which can be improved with training on the multilingual corpus. We also find that increasing model size helps a better understanding of social values, but smaller models can be enhanced by using synthetic data. Our analysis reveals valuable insights into the design methodology of LLMs in connection with their understanding of cultural values.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
