因果干预揭示多语言语言模型中类型学组织的句法机制
Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Models
另 1 家 · 查看机构详情
- The University of Texas at Austin（德克萨斯大学奥斯汀分校）
- Saarland University（萨尔大学）
- Leipzig University（莱比锡大学）
- Heidelberg Institute for Theoretical Studies（海德堡理论研究所）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本研究利用机制可解释性技术，在四种多语言语言模型的三种句法结构上证实跨语言机制迁移，且迁移程度与语言类型学相似度正相关，为语言学理论提供新假说。
中文摘要 AI 辅助
语言学理论长期以来已认识到跨语言的句法规律性，这使得人们提出这些相似结构由相似机制处理的主张。然而，由于我们缺乏对人类处理机制的细粒度、可操作的访问，这一假设难以通过实证检验。本研究利用机制可解释性技术来研究多语言语言模型（multilingual LMs）中的这一问题。我们首先分离出语言内部的机制，随后尝试将这些机制跨语言迁移。在四个模型和三种被广泛研究的结构（主谓数一致、代词性别照应一致以及 filler-gap 宾语提取）上，我们发现一致的跨语言机制迁移现象。我们进一步发现这种迁移是分等级的，类型学上更相似的语言之间存在更多迁移。我们认为，本研究为跨语言句法结构和多语言处理提供了新的假说，更广泛地展示了对语言模型的研究如何能为语言学理论提供参考。
英文摘要
Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar mechanisms. However, this hypothesis has been difficult to test empirically due to our lack of fine-grained, manipulable access of human processing mechanisms. In this work, we take advantage of techniques from mechanistic interpretability to study such a question in multilingual LMs. We first isolate language-internal mechanisms before attempting to transfer them cross-lingually. Across four models and three well-studied constructions (subject--verb number agreement, anaphoric pronoun gender agreement, and filler--gap object extraction) we find consistent cross-lingual mechanism transfer. We further find transfer to be graded, with more transfer between more typologically similar languages. We believe our work provides novel hypotheses about cross-linguistic syntactic structures and multilingual processing, and more broadly shows how the study of language models can help inform linguistic theory.
