多语言性的回响：追踪语言模型微调期间的文化价值转变
The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuning
AI总结：
本文研究多语言语言模型微调时文化价值如何跨语言迁移与改变，并利用训练数据归因识别引发价值转变的数据及语言模式。
摘要：
用不同语言书写的文本反映了其作者不同的、依赖于文化的信念。因此，我们预期在多种语言文本的拼接数据上联合训练的多语言语言模型（MLM）会为每种语言编码不同的文化价值。然而，由于这些语言模型的“多语言性”由跨语言共享所驱动，我们也有理由认为文化价值会从一种语言渗入另一种语言。这限制了 MLM 在实践中的应用，因为除了要能够熟练生成多种语言的文本之外，创建能够服务某个社群的语言技术还要求语言模型的输出对其偏见保持敏感（Naous 等，2023）。然而，人们对文化价值如何在 MLM 中出现和演化仍知之甚少（Hershcovich 等，2022a）。我们通过研究这些价值在微调过程中如何被修正，首次探讨了各语言如何对为不同测试语言编码的文化价值施加影响。聚焦微调阶段使我们能够研究在接触来自不同数据源和语言的新语言经验时，价值转变之间的相互作用。最后，我们使用一种训练数据归因方法，在微调样本及其来源语言中寻找往往会引发价值转变的模式。
英文摘要：
Texts written in different languages reflect different culturally-dependent beliefs of their writers. Thus, we expect multilingual LMs (MLMs), that are jointly trained on a concatenation of text in multiple languages, to encode different cultural values for each language. Yet, as the 'multilinguality' of these LMs is driven by cross-lingual sharing, we also have reason to belief that cultural values bleed over from one language into another. This limits the use of MLMs in practice, as apart from being proficient in generating text in multiple languages, creating language technology that can serve a community also requires the output of LMs to be sensitive to their biases (Naous et al., 2023). Yet, little is known about how cultural values emerge and evolve in MLMs (Hershcovich et al., 2022a). We are the first to study how languages can exert influence on the cultural values encoded for different test languages, by studying how such values are revised during fine-tuning. Focusing on the fine-tuning stage allows us to study the interplay between value shifts when exposed to new linguistic experience from different data sources and languages. Lastly, we use a training data attribution method to find patterns in the fine-tuning examples, and the languages that they come from, that tend to instigate value shifts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
