弥合差距：从英语PLM到马来西亚英语的迁移学习
Bridging the Gap: Transfer Learning from English PLMs to Malaysian English
AI总结：
针对马来西亚英语的低资源克里奥尔语特性，通过微调预训练语言模型MENmBERT和MENBERT，在命名实体识别和关系抽取任务上分别提升1.52%和26.27%的性能。
摘要：
马来西亚英语是一种低资源克里奥尔语，除了标准英语外，还包含马来语、汉语和泰米尔语的元素。由于其独特的形态句法适应、语义特征和代码切换（混合英语和马来语），命名实体识别（NER）模型在从马来西亚英语文本中捕获实体时表现不佳。考虑到这些差距，我们引入了MENmBERT和MENBERT，这是一种具有上下文理解的预训练语言模型，专门针对马来西亚英语定制。我们使用来自马来西亚英语新闻文章（MEN）数据集的手动注释实体和关系对MENmBERT和MENBERT进行了微调。这一微调过程使PLM能够学习捕捉与NER和RE任务相关的马来西亚英语细微差别的表示。与bert-base-multilingual-cased模型相比，MENmBERT在NER和RE任务上分别提高了1.52%和26.27%。尽管NER的整体性能没有显著提升，但我们的进一步分析表明，在12个实体标签评估时，性能有显著提升。这些发现表明，在特定语言和地理区域的语料库上预训练语言模型是提高低资源环境下NER性能的一种有前景的方法。本文发布的数据集和代码为专注于马来西亚英语的NLP研究工作提供了宝贵的资源。
英文摘要：
Malaysian English is a low resource creole language, where it carries the elements of Malay, Chinese, and Tamil languages, in addition to Standard English. Named Entity Recognition (NER) models underperform when capturing entities from Malaysian English text due to its distinctive morphosyntactic adaptations, semantic features and code-switching (mixing English and Malay). Considering these gaps, we introduce MENmBERT and MENBERT, a pre-trained language model with contextual understanding, specifically tailored for Malaysian English. We have fine-tuned MENmBERT and MENBERT using manually annotated entities and relations from the Malaysian English News Article (MEN) Dataset. This fine-tuning process allows the PLM to learn representations that capture the nuances of Malaysian English relevant for NER and RE tasks. MENmBERT achieved a 1.52\% and 26.27\% improvement on NER and RE tasks respectively compared to the bert-base-multilingual-cased model. Although the overall performance of NER does not have a significant improvement, our further analysis shows that there is a significant improvement when evaluated by the 12 entity labels. These findings suggest that pre-training language models on language-specific and geographically-focused corpora can be a promising approach for improving NER performance in low-resource settings. The dataset and code published in this paper provide valuable resources for NLP research work focusing on Malaysian English.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
