压缩的多语言机器翻译模型遗忘了什么？
What Do Compressed Multilingual Machine Translation Models Forget?
AI总结：
本文研究了压缩技术对多语言神经机器翻译模型的影响，发现压缩虽对平均 BLEU 指标影响微小，却导致低资源语言性能显著下降，并放大了性别与语义偏见。
摘要：
最近，超大规模预训练模型在各种自然语言处理（NLP）任务中取得了最先进的结果，但其庞大的规模使得在资源受限的环境中应用它们更具挑战性。压缩技术能够大幅缩减模型规模，进而缩短其推理时间，且对顶级指标的影响微乎其微。然而，在多个任务和/或语言上平均计算出的总体性能可能会掩盖其在代表性不足的特征上出现的急剧性能下降，这可能导致模型所编码偏见的放大。在本研究中，我们通过在不同机器翻译基准（即 FLORES-101、MT-Gender 和 DiBiMT）上对压缩模型进行广泛分析，评估了压缩方法对多语言神经机器翻译模型（MNMT）在不同语系、性别和语义偏见方面的影响。我们表明，代表性不足语言的性能显著下降，而平均 BLEU 指标仅略有降低。有趣的是，通过压缩消除噪声记忆，使得一些中等资源语言的性能获得了显著提升。最后，我们证明压缩放大了内在的性别和语义偏见，甚至在资源丰富的语言中也是如此。代码：https://github.com/alirezamshi/bias-compressedMT
英文摘要：
Recently, very large pre-trained models achieve state-of-the-art results in various natural language processing (NLP) tasks, but their size makes it more challenging to apply them in resource-constrained environments. Compression techniques allow to drastically reduce the size of the models and therefore their inference time with negligible impact on top-tier metrics. However, the general performance averaged across multiple tasks and/or languages may hide a drastic performance drop on under-represented features, which could result in the amplification of biases encoded by the models. In this work, we assess the impact of compression methods on Multilingual Neural Machine Translation models (MNMT) for various language groups, gender, and semantic biases by extensive analysis of compressed models on different machine translation benchmarks, i.e. FLORES-101, MT-Gender, and DiBiMT. We show that the performance of under-represented languages drops significantly, while the average BLEU metric only slightly decreases. Interestingly, the removal of noisy memorization with compression leads to a significant improvement for some medium-resource languages. Finally, we demonstrate that compression amplifies intrinsic gender and semantic biases, even in high-resource languages. Code: https://github.com/alirezamshi/bias-compressedMT
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
