发表机构
Boston University; Technische Universität Clausthal（波士顿大学; 克劳斯塔尔工业大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出语际假设，即LLMs通过潜在任务无关特征空间翻译，通过BLEU分数、模型组件因果性及单语言微调三条证据支持，为LLMs翻译机制提供新理解。
AI 中文摘要
大语言模型（LLMs）近期在机器翻译任务上展现出优于强大监督基线的性能，这引发了关于LLMs执行跨语言翻译的底层机制的疑问。受近期可解释性研究结果的启发——即LLMs利用大规模多语言潜在特征表示来执行语言建模——本文提出了语际假设。该假设认为，语言模型通过将源句子读入潜在特征空间，并从该潜在特征空间中读取以生成目标句子，从而完成翻译。本文提供了三条支持该假设的证据：（1）不同语言对之间的BLEU分数差异，可在很大程度上通过特定语言的能力预测，且不存在特定语言对的交互项；（2）许多模型组件在单语言任务和翻译任务中均具有因果影响力；（3）在单语言数据上进行微调，相对于在对齐文档上进行微调，可恢复大部分翻译性能提升。这些证据共同为语际假设提供了汇聚性支持，并为理解和改进LLMs在翻译任务中的应用提供了新的思路。
英文摘要
Large language models (LLMs) have recently demonstrated improved machine translation performance over strong supervised baselines. This raises questions as to what mechanisms underlie how LLMs perform machine translation between languages. Motivated by recent interpretability findings--namely, that LLMs use massively multilingual latent feature representations to perform language modeling--we propose the interlingua hypothesis. The hypothesis holds that language models translate by reading a source sentence into a latent feature space, and generate a target sentence by reading from the latent feature space. We show three lines of evidence in support of this hypothesis: (1) variance in BLEU across language pairs is largely predictable from language-specific competences with no language pair-specific interaction terms; (2) many model components are causally influential in both monolingual tasks and translation tasks; and (3) fine-tuning on monolingual data recovers a large proportion of translation improvements relative to fine-tuning on aligned documents. Together, these provide convergent evidence in support of the interlingua hypothesis, and suggest new ways of understanding and improving how LLMs can be leveraged to perform translation tasks.
Comments21 pages, 15 figures, 11 tables
