面向多语言机器翻译的语言特定层学习方法
Learning Language-Specific Layers for Multilingual Machine Translation
AI总结：
针对多语言机器翻译中新增语言会降低单语言模型容量的问题，提出语言特定Transformer层方法，通过神经架构搜索确定层的放置方式，在不增加前向传播计算量与参数量的前提下提升了翻译性能。
摘要：
多语言机器翻译有望提升非英语语言间的翻译质量，其优势体现在多方面：即更低的延迟（无需二次翻译），以及更少的错误级联（例如避免经英语转译时丢失性别和正式程度信息）。但缺点是，加入更多语言会降低每种语言对应的模型容量，通常的应对方式是增大整体模型规模，这会提升训练难度并减慢推理速度。本研究提出了语言特定Transformer层（Language-Specific Transformer Layers，简称LSLs），该方法能够在保持前向传播计算量与参数量不变的前提下，提升模型容量。其核心思路是让编码器的部分层为源语言或目标语言特定层，其余层则保持共享。我们采用受神经架构搜索启发的方法，研究了这些层的最优放置方式；在独立解码器架构上，相比不使用LSLs的模型，该方法取得了1.3 chrF（1.5 spBLEU）分的提升，在共享解码器架构上则取得了1.9 chrF（2.2 spBLEU）分的提升。
英文摘要：
Multilingual Machine Translation promises to improve translation quality between non-English languages. This is advantageous for several reasons, namely lower latency (no need to translate twice), and reduced error cascades (e.g., avoiding losing gender and formality information when translating through English). On the downside, adding more languages reduces model capacity per language, which is usually countered by increasing the overall model size, making training harder and inference slower. In this work, we introduce Language-Specific Transformer Layers (LSLs), which allow us to increase model capacity, while keeping the amount of computation and the number of parameters used in the forward pass constant. The key idea is to have some layers of the encoder be source or target language-specific, while keeping the remaining layers shared. We study the best way to place these layers using a neural architecture search inspired approach, and achieve an improvement of 1.3 chrF (1.5 spBLEU) points over not using LSLs on a separate decoder architecture, and 1.9 chrF (2.2 spBLEU) on a shared decoder one.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
