发表机构
Saarland University; German Research Center for Artificial Intelligence (DFKI); Centre for European Research in Trusted AI (CERTAIN); University of Göttingen（萨尔大学; 德国人工智能研究中心; 欧洲可信人工智能研究中心; 哥廷根大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究揭示多语言大语言模型的翻译可进一步分离为句法与表层语言过程，通过构建受控数据集等方法发现模型先构建目标侧词序，识别出句法敏感的注意力头，完善了翻译机制的分解结论。
AI 中文摘要
多语言大语言模型（mLLM）在机器翻译中表现出色，但人们对其将表示从一种语言转换为另一种语言的机制仍不完全了解。先前研究表明，翻译在mLLM内可分解为可分离的过程：首先独立表示概念内容，再生成特定语言形式。本研究显示，翻译的模块化程度远超此前假设，翻译过程中的输出语言生成实际可进一步分离为句法和表层语言过程。我们构建了受控多语言数据集以分离词序的跨语言差异，并使用因果干预和探测方法追踪翻译过程中表示的转换。研究发现，模型会先构建目标侧词序，再实现目标语言表层形式；还识别出对句法转换具有选择性敏感性、但对语言身份基本不变的独立注意力头。这些结果确立了句法结构是翻译的独立阶段，扩展了先前的分解结论，阐明了翻译如何由mLLM内功能不同的组件实现。
英文摘要
Multilingual large language models (mLLMs) achieve strong performance in machine translation, yet our understanding of the mechanisms by which they transform representations from one language to another remains incomplete. Prior work suggests that translation decomposes into separable processes within an mLLM, where conceptual content is first represented independently, followed by a production into language-specific form. In this work, we show that translation is even more modular than previously assumed and that the output language production in translation processes is actually further separable into a syntax and a surface language process. We construct controlled multilingual datasets that isolate cross-linguistic differences in word-order and use causal interventions and probing to track how representations are transformed during translation. We find that models first construct target-side word-order before realizing the target language surface form. We identify individual attention heads that are selectively sensitive to syntactic transformations while remaining largely invariant to language identity. These results establish the commitment to a syntactic structure as an independent stage in translation, extending prior decompositions and showing how translation is implemented by functionally different components within mLLMs.
CommentsAccepted to EMNLP Findings 2026
