arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.16030cs.CLcs.AI

面向低资源语言的基于音素表示的零样本跨语言命名实体识别

Zero-Shot Cross-Lingual NER Using Phonemic Representations for Low-Resource Languages

GIST(光州科学技术院);Korea University(高丽大学);Carnegie Mellon University(卡内基梅隆大学);Sogang University(西江大学)

Jimin Sohn, Haeji Jung, Alex Cheng, Jooeon Kang, Yilin Du, David R. Mortensen

AI总结:

本文提出基于国际音标(IPA)音素表示的零样本跨语言命名实体识别方法,以解决低资源语言先验知识不足的问题,实验表明该方法在极低资源语言上显著优于基线模型,尤其对非拉丁文字具有鲁棒性。

摘要:

现有的零样本跨语言命名实体识别方法需要对目标语言有大量的先验知识,这对低资源语言来说并不现实。本文提出了一种新的命名实体识别方法,使用基于国际音标(IPA)的音素表示来弥合不同语言表示之间的差距。实验表明,我们的方法在极低资源语言上显著优于基线模型,取得了最高的平均F1分数(46.38%)和最低的标准差(12.67),尤其展示了其在非拉丁文字上的鲁棒性。代码可在 https://github.com/Gabriel819/zeroshot_ner.git 获取。

英文摘要:

Existing zero-shot cross-lingual NER approaches require substantial prior knowledge of the target language, which is impractical for low-resource languages. In this paper, we propose a novel approach to NER using phonemic representation based on the International Phonetic Alphabet (IPA) to bridge the gap between representations of different languages. Our experiments show that our method significantly outperforms baseline models in extremely low-resource languages, with the highest average F1 score (46.38%) and lowest standard deviation (12.67), particularly demonstrating its robustness with non-Latin scripts. Our codes are available at https://github.com/Gabriel819/zeroshot_ner.git

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑