arXiv 2308.142802023-09-11cs.CLcs.AI

FonMTL：面向Fon语的多任务学习

FonMTL: Towards Multitask Learning for the Fon Language

Carnegie Mellon University Africa（卡内基梅隆大学非洲分校）；Masakhane NLP（Masakhane自然语言处理）；Mila Quebec AI Institute（魁北克人工智能研究所Mila）；McGill University（麦吉尔大学）；Lelapa AI（Lelapa人工智能）；Lanfrica；Microsoft AI For Good Kenya（微软肯尼亚AI向善部门）

Bonaventure F. P. Dossou, Iffanice Houndayi, Pamely Zantou, Gilles Hacheme

备注：: Accepted at WiNLP workshop, co-located at EMNLP 2023

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对低资源Fon语，首次提出多任务学习框架，通过共享编码器同时处理NER与POS任务，性能优于单任务微调的多语言模型，并发现等权重损失策略效果最佳。

摘要：

Fon语约有200万人使用，是一种真正低资源的非洲语言，其在线存在有限，且现有数据集也寥寥无几。多任务学习是一种旨在通过在不同但相关的任务之间共享知识来提升模型泛化能力的学习范式：这在数据极度稀缺的场景中可能尤为普遍。在本文中，我们提出了面向Fon语自然语言处理中模型能力增强的首次多任务学习探索方法。具体而言，我们探索了Fon语的两个任务：命名实体识别（NER）和词性标注（POS）。我们利用两个语言模型头作为编码器来构建输入的共享表示，并使用线性层块进行每个任务的分类。我们在Fon语NER和POS任务上的结果表明，与在单一任务上微调的几个多语言预训练语言模型相比，其性能具有竞争力（或更好）。此外，我们进行了几项消融研究，以评估两种不同损失组合策略的效率，并发现在我们的案例中，等权重损失加权方法效果最佳。我们的代码已在 https://github.com/bonaventuredossou/multitask_fon 开源。

英文摘要：

The Fon language, spoken by an average 2 million of people, is a truly low-resourced African language, with a limited online presence, and existing datasets (just to name but a few). Multitask learning is a learning paradigm that aims to improve the generalization capacity of a model by sharing knowledge across different but related tasks: this could be prevalent in very data-scarce scenarios. In this paper, we present the first explorative approach to multitask learning, for model capabilities enhancement in Natural Language Processing for the Fon language. Specifically, we explore the tasks of Named Entity Recognition (NER) and Part of Speech Tagging (POS) for Fon. We leverage two language model heads as encoders to build shared representations for the inputs, and we use linear layers blocks for classification relative to each task. Our results on the NER and POS tasks for Fon, show competitive (or better) performances compared to several multilingual pretrained language models finetuned on single tasks. Additionally, we perform a few ablation studies to leverage the efficiency of two different loss combination strategies and find out that the equal loss weighting approach works best in our case. Our code is open-sourced at https://github.com/bonaventuredossou/multitask_fon.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。