无需训练却有增益：用于增强免训练语言适配器的 Language Arithmetic
No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement
AI总结：
本文提出 Language Arithmetic，通过扩展 task arithmetic 对语言适配器做免训练加法后处理，在基于 MAD-X 的三类跨语言任务中显著提升基线，尤其改善零样本效果。
摘要：
模块化深度学习是缓解多语言诅咒、防止负面干扰影响并在多语言预训练语言模型中实现跨语言性能的最先进解决方案。然而，这种方法的一个权衡是减少了来自亲缘关系密切语言的正迁移学习。为此，我们提出了一种名为 language arithmetic 的新方法，它支持免训练后处理以解决这一局限。该方法扩展了 task arithmetic 框架，将通过加法进行学习应用于语言适配器，使该框架从多任务设置转向多语言设置。在基于 MAD-X 的一组跨语言方案中，所提解决方案作为后处理流程，在三个下游任务上证明了有效性。Language arithmetic 持续改进基线并取得显著增益，尤其是在最具挑战性的零样本应用情形中。我们的代码和模型可在 https://github.com/mklimasz/language-arithmetic 获取。
英文摘要：
Modular deep learning is the state-of-the-art solution for lifting the curse of multilinguality, preventing the impact of negative interference and enabling cross-lingual performance in Multilingual Pre-trained Language Models. However, a trade-off of this approach is the reduction in positive transfer learning from closely related languages. In response, we introduce a novel method called language arithmetic, which enables training-free post-processing to address this limitation. Extending the task arithmetic framework, we apply learning via addition to the language adapters, transitioning the framework from a multi-task to a multilingual setup. The effectiveness of the proposed solution is demonstrated on three downstream tasks in a MAD-X-based set of cross-lingual schemes, acting as a post-processing procedure. Language arithmetic consistently improves the baselines with significant gains, especially in the most challenging case of zero-shot application. Our code and models are available at https://github.com/mklimasz/language-arithmetic .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
