文化知识的跨语言迁移:一种非对称现象
Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon
AI总结:
本文通过可解释框架研究LLM语言适应中的文化知识跨语言迁移,发现英语与高资源语言双向迁移、低资源语言单向流向英语的非对称现象,并提出频率假设解释该现象。
摘要:
尽管已有大量研究评估大语言模型(LLMs)处理全球文化多样性的能力,但其文化知识获取机制,尤其是在多语言环境下的机制,仍不清楚。我们通过研究语言适应过程中文化知识如何跨语言迁移来探讨这一问题。我们引入了一个可解释的框架来研究这种迁移,确保训练数据透明并控制迁移效应。通过对四种非英语文化的研究,我们观察到英语与其他高资源语言之间存在双向文化迁移,而低资源语言主要向英语迁移知识,反向流动有限。为解释这一非对称现象,我们提出一个基于频率的假设:在预训练数据中出现频率更高的文化知识更容易迁移,这一假设得到了对训练语料实证分析的支持。
英文摘要:
Despite substantial research efforts evaluating how well large language models~(LLMs) handle global cultural diversity, the mechanisms behind their cultural knowledge acquisition, particularly in multilingual settings, remain unclear. We study this question by investigating how cultural knowledge transfers across languages during language adaptation of LLMs. We introduce an interpretable framework for studying this transfer, ensuring training data transparency and controlling transfer effects. Through a study of four non-Anglophonic cultures, we observe bidirectional cultural transfer between English and other high-resource languages, while low-resource languages primarily transfer knowledge to English with limited reverse flow. To explain this asymmetric phenomenon, we propose a frequency-based hypothesis: cultural knowledge appearing more frequently in the pretraining data transfers more easily, which is supported by empirical analysis of the training corpora.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。