发表机构
Harvard University; University of Zaragoza; Boston University; Univ Toulon; Aix Marseille Univ; CNRS; LIS; ILLS（哈佛大学; 萨拉戈萨大学; 波士顿大学; 土伦大学; 艾克斯-马赛大学; 法国国家科学研究中心; 信息科学实验室; 语言与语言科学研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究多语言大语言模型的跨语言共享机制，以主谓一致为对象，通过29种语言的实验发现其复用部分共享计算结构，且与屈折语言的电路更相似。
AI 中文摘要
多语言大语言模型（multilingual large language models）常可跨语言泛化，已有研究表明其内部机制可跨语言重叠，但目前仍不清楚这种共享何时出现，以及是否随同一语法操作的显性实现而变化。本文针对现在时主谓一致（一种在不同语言间差异显著且在英语中仅弱表达的形态句法过程）研究该问题。通过在29种语言和5个开源模型家族中使用激活修补（activation patching）和注意力分析，本文识别出与一致现象存在因果关联的注意力头，并跨语言比较这些头级特征。研究发现，具有显性人称/数屈折变化的语言，其一致电路比非屈折语言更相似；当分析聚焦于恢复屈折对立本身时，共享程度最高。英语作为一种兼具信息性的桥梁案例，恰好会在需要显性一致的语境中变得与屈折语言更相似。最后，许多相关注意力头在不同语言间展现出相似的注意力模式，表明跨语言重叠既反映了共享的功能角色，也反映了共享的定位。总体而言，这些结果表明，多语言大语言模型为形态句法一致复用了部分共享的计算结构，而非依赖完全独立的特定语言解决方案。
英文摘要
Multilingual large language models often generalize across languages, and prior work suggests that their internal mechanisms can overlap cross-lingually. It remains unclear, however, when such sharing emerges and whether it varies with the overt realization of the same grammatical operation. We investigate this question for present-tense subject-verb agreement, a morphosyntactic process that varies substantially across languages and is only weakly expressed in English. Using activation patching and attention analysis across 29 languages and five open-source model families, we identify the attention heads causally implicated in agreement and compare these head-level signatures across languages. We find that languages with overt person/number inflection exhibit more similar agreement circuitry than non-conjugating languages, with the strongest sharing appearing when the analysis isolates recovery of the inflectional contrast itself. English provides an informative bridge case, becoming more similar to conjugating languages precisely in contexts where overt agreement is required. Finally, many implicated heads display similar attention patterns across languages, suggesting that cross-lingual overlap reflects shared functional roles as well as shared localization. Together, these results indicate that multilingual LLMs reuse partially shared computational structure for morphosyntactic agreement rather than relying on fully separate language-specific solutions.
Comments25 pages including appendices, 16 figures. Accepted to COLM 2026
