语言适配器在自然语言理解跨语言迁移中的影响
The Impact of Language Adapters in Cross-Lingual Transfer for NLU
AI总结：
本文通过消融实验研究语言适配器在零样本跨语言NLU迁移中的作用，发现其影响不一致，保留源语言适配器常更优，移除后影响微弱。
摘要：
模块化深度学习已被提出用于高效地将预训练模型适配到新任务、新领域和新语言。特别是，将语言适配器与任务适配器相结合，在某种语言不存在监督数据的情况下显示出潜力。在本文中，我们探讨了语言适配器在自然语言理解（NLU）基准的零样本跨语言迁移中的作用。我们通过详细消融研究，使用两个多语言模型和三个多语言数据集，考察了包含目标语言适配器的影响。我们的结果表明，目标语言适配器的影响在不同任务、语言和模型之间高度不一致。保留源语言适配器往往能带来相当的性能，有时甚至更好。训练后移除语言适配器仅产生微弱的负面影响，表明语言适配器对预测结果没有强烈影响。
英文摘要：
Modular deep learning has been proposed for the efficient adaption of pre-trained models to new tasks, domains and languages. In particular, combining language adapters with task adapters has shown potential where no supervised data exists for a language. In this paper, we explore the role of language adapters in zero-shot cross-lingual transfer for natural language understanding (NLU) benchmarks. We study the effect of including a target-language adapter in detailed ablation studies with two multilingual models and three multilingual datasets. Our results show that the effect of target-language adapters is highly inconsistent across tasks, languages and models. Retaining the source-language adapter instead often leads to an equivalent, and sometimes to a better, performance. Removing the language adapter after training has only a weak negative effect, indicating that the language adapters do not have a strong impact on the predictions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
