GradSim：基于梯度的语言分组用于有效的多语言训练
GradSim: Gradient-Based Language Grouping for Effective Multilingual Training
AI总结：
本文提出GradSim，一种基于梯度相似性的语言分组方法，用于多语言训练，有效避免语言间的负面干扰，并在AfriSenti等基准上取得最优性能。
摘要：
世界上大多数语言对自然语言处理模型构成了低资源挑战。通过多语言训练，知识可以在语言之间共享。然而，并非所有语言都能相互产生积极影响，如何为多语言训练选择最合适的语言集合，并避免在特征或数据分布不兼容的语言之间产生负面干扰，仍是一个开放的研究问题。在本文中，我们提出了GradSim，一种基于梯度相似性的语言分组方法。我们在三个不同的多语言基准数据集上的实验表明，与其他相似性度量相比，它能带来最大的性能提升，并且与跨语言模型性能的相关性更好。因此，我们在AfriSenti（一个用于低资源非洲语言情感分析的基准数据集）上创造了新的最先进水平。在我们的深入分析中，我们进一步揭示，除了语言特征外，数据集的主题对语言分组也起着重要作用，并且transformer模型的较低层编码语言特定特征，而较高层捕获任务特定信息。
英文摘要：
Most languages of the world pose low-resource challenges to natural language processing models. With multilingual training, knowledge can be shared among languages. However, not all languages positively influence each other and it is an open research question how to select the most suitable set of languages for multilingual training and avoid negative interference among languages whose characteristics or data distributions are not compatible. In this paper, we propose GradSim, a language grouping method based on gradient similarity. Our experiments on three diverse multilingual benchmark datasets show that it leads to the largest performance gains compared to other similarity measures and it is better correlated with cross-lingual model performance. As a result, we set the new state of the art on AfriSenti, a benchmark dataset for sentiment analysis on low-resource African languages. In our extensive analysis, we further reveal that besides linguistic features, the topics of the datasets play an important role for language grouping and that lower layers of transformer models encode language-specific features while higher layers capture task-specific information.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
