论双语语言模型中共享语法表征的习得
On the Acquisition of Shared Grammatical Representations in Bilingual Language Models
AI总结：
通过控制数据量和语言接触顺序训练小型双语模型，并利用结构启动方法，揭示跨语言语法共享表征的不对称性及其在类型学差异较大语言上的局限性。
摘要：
跨语言迁移对当代语言模型的多语言能力至关重要，但其发生机制尚不十分清楚。我们探究当一个单语语言模型开始接受第二语言训练时会发生什么。具体而言，我们训练小型双语模型，并控制每种语言的数据量以及语言接触的顺序。为了寻找共享多语言表征的证据，我们采用结构启动这一用于研究人类语法表征的方法。我们首先复现了先前的跨语言结构启动结果，并发现在控制训练数据量和语言接触顺序后，不同语言对和方向上存在不对称效应。我们认为这种不对称性可能为关于人类结构启动效应的假设提供启发。我们还发现，对于相似度较低的语言对，结构启动效应较不稳健，这凸显了跨语言迁移学习和共享表征在类型学差异较大的语言上可能存在的局限性。
英文摘要：
Crosslingual transfer is crucial to contemporary language models' multilingual capabilities, but how it occurs is not well understood. We ask what happens to a monolingual language model when it begins to be trained on a second language. Specifically, we train small bilingual models for which we control the amount of data for each language and the order of language exposure. To find evidence of shared multilingual representations, we turn to structural priming, a method used to study grammatical representations in humans. We first replicate previous crosslingual structural priming results and find that after controlling for training data quantity and language exposure, there are asymmetrical effects across language pairs and directions. We argue that this asymmetry may shape hypotheses about human structural priming effects. We also find that structural priming effects are less robust for less similar language pairs, highlighting potential limitations of crosslingual transfer learning and shared representations for typologically diverse languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
