测量多语言Transformer在句子分类上的跨语言迁移能力
Measuring Cross-Lingual Transferability of Multilingual Transformers on Sentence Classification
AI总结：
提出IGap指标，用于测量多语言Transformer在句子分类上的跨语言迁移能力，实验证明其优于基线，并揭示三个相关发现。
摘要：
近期研究展示了预训练多语言Transformer的显著能力，尤其是跨语言迁移能力。然而，当前方法未能很好地测量跨语言迁移能力，阻碍了对多语言Transformer的理解。本文提出IGap，一种用于多语言Transformer在句子分类任务上的跨语言迁移能力度量指标。IGap考虑了训练误差，并且无需端任务数据即可估计迁移能力。实验结果表明，IGap在迁移能力测量和迁移方向排序方面优于基线指标。此外，我们进行了广泛的系统性实验，比较了不同多语言Transformer、微调算法和迁移方向之间的迁移能力。更重要的是，我们的结果揭示了关于跨语言迁移的三个发现，有助于更好地理解多语言Transformer。
英文摘要：
Recent studies have exhibited remarkable capabilities of pre-trained multilingual Transformers, especially cross-lingual transferability. However, current methods do not measure cross-lingual transferability well, hindering the understanding of multilingual Transformers. In this paper, we propose IGap, a cross-lingual transferability metric for multilingual Transformers on sentence classification tasks. IGap takes training error into consideration, and can also estimate transferability without end-task data. Experimental results show that IGap outperforms baseline metrics for transferability measuring and transfer direction ranking. Besides, we conduct extensive systematic experiments where we compare transferability among various multilingual Transformers, fine-tuning algorithms, and transfer directions. More importantly, our results reveal three findings about cross-lingual transfer, which helps us to better understand multilingual Transformers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
