arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2305.00090cs.CL

NLNDE at SemEval-2023 Task 12:面向低资源多语言情感分析的自适应预训练与源语言选择

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

Bosch Center for Artificial Intelligence(博世人工智能中心);Center for Information and Language Processing (CIS), LMU Munich(慕尼黑大学语言信息处理中心);Karlsruhe University of Applied Sciences(卡尔斯鲁厄应用科学大学)

Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, Hinrich Schütze

AI总结:

本文针对低资源非洲语言情感分析,提出利用语言与任务自适应预训练及源语言选择策略,显著提升了跨语言迁移效果,在SemEval-2023任务中赢得15个赛道中的8个并取得多语言评估最佳。

摘要:

本文介绍了我们为 SemEval-2023 Task 12“使用 Twitter 数据集进行低资源非洲语言情感分析”开发的系统。情感分析是自然语言处理中研究最广泛的应用之一。然而,大多数先前的工作仍集中在少数高资源语言上。由于该任务中训练数据有限,为低资源语言构建可靠的情感分析系统仍然具有挑战性。在这项工作中,我们提出在非洲文本上利用语言自适应和任务自适应预训练,并在以非洲语言为中心的预训练语言模型之上研究带有源语言选择的迁移学习。我们的主要发现是:(1)使用小型但相关的语料库使预训练模型适应目标语言和任务,可将性能显著提高超过 10 个 F1 分数点。(2)在训练期间选择具有正向迁移收益的源语言可以避免来自不相似语言的有害干扰,从而在多语言和跨语言设置中获得更好的结果。在共享任务中,我们的系统赢得了 15 个赛道中的 8 个,并且在多语言评估中表现最佳。

英文摘要:

This paper describes our system developed for the SemEval-2023 Task 12 "Sentiment Analysis for Low-resource African Languages using Twitter Dataset". Sentiment analysis is one of the most widely studied applications in natural language processing. However, most prior work still focuses on a small number of high-resource languages. Building reliable sentiment analysis systems for low-resource languages remains challenging, due to the limited training data in this task. In this work, we propose to leverage language-adaptive and task-adaptive pretraining on African texts and study transfer learning with source language selection on top of an African language-centric pretrained language model. Our key findings are: (1) Adapting the pretrained model to the target language and task using a small yet relevant corpus improves performance remarkably by more than 10 F1 score points. (2) Selecting source languages with positive transfer gains during training can avoid harmful interference from dissimilar languages, leading to better results in multilingual and cross-lingual settings. In the shared task, our system wins 8 out of 15 tracks and, in particular, performs best in the multilingual evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑