一个模型来翻译它们全部？多语言模型融合的探索之旅
One Model to Translate Them All? A Journey to Mount Doom for Multilingual Model Merging
AI总结：
本文研究了多语言机器翻译中权重空间模型融合的行为，发现融合会降低性能，尤其在目标语言差异时。通过分析内部表示发现语言特定神经元集中在嵌入层和上层Transformer块，而中间层在多语言间共享。细调使语言选择性重新分布，影响生成层的表示差异。
摘要：
权重空间模型融合通过独立微调模型而不访问原始训练数据，提供了一种联合训练的实用替代方案。虽然在多任务场景中融合表现良好，但其在多语言环境中的行为仍不明确。我们通过在大规模双语语料上完全微调语言模型，并评估标准融合策略，系统研究了多语言机器翻译中的权重空间融合。实验发现，融合会降低性能，尤其是在目标语言差异时。为解释这一失败，我们利用跨度条件神经元选择性和分层中心核对齐分析内部表示。我们发现语言特定的神经元集中在嵌入层和上层Transformer块，而中间层在多语言间保持 largely 共享。关键的是，微调会重新分布而非锐化语言选择性：监督和相关语言的神经元变得不那么排斥，而无监督语言的神经元则更加孤立。这种重新分布增加了更高层的表示分歧，这些高层控制生成。这些发现表明，多语言微调可能以减少与标准权重空间融合假设兼容性的方式重塑几何结构。我们的工作因此提供了解释为何在多语言翻译场景中融合失败的原因。
英文摘要：
Weight-space model merging combines independently fine-tuned checkpoints without access to the original training data. While merging has shown promise in multitask settings, its behavior in multilingual generative systems remains underexplored. We systematically study weight-space merging for multilingual machine translation by fully fine-tuning language models on large-scale bilingual corpora and evaluating representative merging strategies across shared-source, shared-target, and bidirectional consolidation settings. Our experiments reveal a strong directional asymmetry. Merging is comparatively more effective when models share a target language, improving multilingual coverage over the base model, but it still fails to preserve the peak performance of language-specific checkpoints. In contrast, when target languages differ, performance degrades sharply, especially in shared-source and bidirectional settings. To explain this behavior, we analyze internal representations and find that fine-tuning does not create disjoint language-specific sub-networks. Instead, independently fine-tuned models activate largely overlapping neurons while reshaping upper-layer target-generation representations into incompatible geometries. These findings suggest that multilingual merging failures arise from target-side geometric misalignment within shared computational units, challenging the assumptions underlying standard weight-space merging for multilingual translation. We make the code publicly available at https://github.com/babangain/mt-model-merging
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
