使用目标语言就绪任务适配器的跨语言迁移
Cross-Lingual Transfer with Target Language-Ready Task Adapters
AI总结：
针对 MAD-X 任务适配器训练与推理时语言适配器不匹配的问题，本文提出目标语言就绪（TLR）适配器，在保持模块化的同时提升跨语言迁移性能。
摘要：
适配器已成为一种模块化且参数高效的（零样本）跨语言迁移方法。成熟的 MAD-X 框架采用相互独立的语言适配器和任务适配器，可任意组合，以将任意任务迁移到任意目标语言。随后，作为 MAD-X 框架扩展的 BAD-X 通过创建特定于源语言—目标语言对的“双语”适配器，实现了更好的迁移效果，但代价是牺牲了 MAD-X 的模块化。在本研究中，我们旨在兼得二者之长：（i）微调适配于目标语言的任务适配器（即所谓的“目标语言就绪”（TLR）适配器），以保持较高的迁移性能；同时（ii）不牺牲 MAD-X 高度模块化的设计。“目标语言就绪”适配器的核心思想是解决 MAD-X 的训练与推理之间的差异：任务适配器在推理期间才第一次“看到”目标语言适配器，因此可能与其并不完全兼容。我们通过在训练期间让任务适配器接触目标语言适配器来解决这种不匹配，并通过实验验证了该思想的若干变体：在最简单的形式中，我们在任务适配器训练期间交替使用源语言适配器和目标语言适配器，这一做法还可推广到在任意一组语言适配器之间循环。我们在一系列标准跨语言基准上评估了具有不同通用程度、基于 TLR 的不同迁移配置，发现最通用（因而也最模块化）的配置在大多数任务和语言上始终优于 MAD-X 和 BAD-X。
英文摘要：
Adapters have emerged as a modular and parameter-efficient approach to (zero-shot) cross-lingual transfer. The established MAD-X framework employs separate language and task adapters which can be arbitrarily combined to perform the transfer of any task to any target language. Subsequently, BAD-X, an extension of the MAD-X framework, achieves improved transfer at the cost of MAD-X's modularity by creating "bilingual" adapters specific to the source-target language pair. In this work, we aim to take the best of both worlds by (i) fine-tuning task adapters adapted to the target language(s) (so-called "target language-ready" (TLR) adapters) to maintain high transfer performance, but (ii) without sacrificing the highly modular design of MAD-X. The main idea of "target language-ready" adapters is to resolve the training-vs-inference discrepancy of MAD-X: the task adapter "sees" the target language adapter for the very first time during inference, and thus might not be fully compatible with it. We address this mismatch by exposing the task adapter to the target language adapter during training, and empirically validate several variants of the idea: in the simplest form, we alternate between using the source and target language adapters during task adapter training, which can be generalized to cycling over any set of language adapters. We evaluate different TLR-based transfer configurations with varying degrees of generality across a suite of standard cross-lingual benchmarks, and find that the most general (and thus most modular) configuration consistently outperforms MAD-X and BAD-X on most tasks and languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
