arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.09717cs.CL
收藏

UniBridge:低资源语言跨语言迁移学习的统一方法

UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages

VinAI Research(VinAI研究院);Nanyang Technological University(南洋理工大学);Ho Chi Minh City University of Technology (HCMUT)(胡志明市理工大学);VNU-HCM(胡志明市国家大学)

Trinh Pham, Khoi M. Le, Luu Anh Tuan

AI总结:

本文提出UniBridge,通过优化嵌入初始化和词汇表大小,提升低资源语言的跨语言迁移学习效果,实验显示F1分数显著提高。

摘要:

在本文中,我们介绍了UniBridge(具有优化嵌入和词汇表的跨语言迁移学习),这是一种旨在提高跨语言迁移学习效果的综合方法,尤其适用于资源有限的语言。我们的方法解决了语言模型的两个关键要素:嵌入的初始化和最优词汇表大小。具体来说,我们提出了一种新颖的嵌入初始化方法,该方法利用词汇和语义对齐来实现语言对齐。此外,我们提出了一种系统搜索最优词汇表大小的方法,确保模型复杂度和语言覆盖之间的平衡。我们在多语言数据集上的实验表明,我们的方法显著提高了多种语言的F1分数。UniBridge是适用于各种语言跨语言系统的稳健且适应性强的解决方案,强调了在跨语言环境中初始化嵌入和选择合适词汇表大小的重要性。

英文摘要:

In this paper, we introduce UniBridge (Cross-Lingual Transfer Learning with Optimized Embeddings and Vocabulary), a comprehensive approach developed to improve the effectiveness of Cross-Lingual Transfer Learning, particularly in languages with limited resources. Our approach tackles two essential elements of a language model: the initialization of embeddings and the optimal vocabulary size. Specifically, we propose a novel embedding initialization method that leverages both lexical and semantic alignment for a language. In addition, we present a method for systematically searching for the optimal vocabulary size, ensuring a balance between model complexity and linguistic coverage. Our experiments across multilingual datasets show that our approach greatly improves the F1-Score in several languages. UniBridge is a robust and adaptable solution for cross-lingual systems in various languages, highlighting the significance of initializing embeddings and choosing the right vocabulary size in cross-lingual environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑