BanglaCoNER：面向健壮的孟加拉语复杂命名实体识别
BanglaCoNER: Towards Robust Bangla Complex Named Entity Recognition
摘要：
命名实体识别（NER）是自然语言处理中的一项基本任务，涉及对文本中的命名实体进行识别和分类。但是，尽管孟加拉语是全球第七大语言，但在复杂的命名实体识别方面还没有做很多工作。CNER是一项比传统NER更具挑战性的任务，因为它涉及识别和分类复杂和复合实体，这在孟加拉语中并不常见。在本文中，我们提出了孟加拉复杂命名实体识别挑战赛的获奖解决方案-使用两种不同的方法解决BanglaCoNER数据集上的CNER任务，即条件随机场（CRF）和基于Transformer的深度学习模型（如BanglaBERT）的微调。该数据集由15300个用于训练的句子和800个用于验证的句子组成，格式为.conll。对数据集的探索性数据分析（EDA）显示，该数据集有7个不同的NER标签，明显存在英语单词，表明该数据集是合成的，可能是翻译的产物。我们尝试了各种特征组合，包括词性（POS）标签，单词后缀，地名词典和嵌入的聚类信息，同时还微调了NER的BanglaBERT（大型）模型。我们发现，并非所有的语言模式对人类来说都是显而易见的，甚至是直观的，这就是为什么基于深度学习的模型被证明是NLP中更有效的模型，包括CNER任务。我们微调的BanglaBERT（大型）模型在验证集上的F1得分为0.79。总的来说，我们的研究强调了孟加拉语复杂命名实体识别的重要性，特别是在合成数据集的背景下。我们的研究结果还证明了深度学习模型的有效性，例如孟加拉语中NER的BanglaBERT。
英文摘要：
Named Entity Recognition (NER) is a fundamental task in natural language processing that involves identifying and classifying named entities in text. But much work hasn't been done for complex named entity recognition in Bangla, despite being the seventh most spoken language globally. CNER is a more challenging task than traditional NER as it involves identifying and classifying complex and compound entities, which are not common in Bangla language. In this paper, we present the winning solution of Bangla Complex Named Entity Recognition Challenge - addressing the CNER task on BanglaCoNER dataset using two different approaches, namely Conditional Random Fields (CRF) and finetuning transformer based Deep Learning models such as BanglaBERT. The dataset consisted of 15300 sentences for training and 800 sentences for validation, in the .conll format. Exploratory Data Analysis (EDA) on the dataset revealed that the dataset had 7 different NER tags, with notable presence of English words, suggesting that the dataset is synthetic and likely a product of translation. We experimented with a variety of feature combinations including Part of Speech (POS) tags, word suffixes, Gazetteers, and cluster information from embeddings, while also finetuning the BanglaBERT (large) model for NER. We found that not all linguistic patterns are immediately apparent or even intuitive to humans, which is why Deep Learning based models has proved to be the more effective model in NLP, including CNER task. Our fine tuned BanglaBERT (large) model achieves an F1 Score of 0.79 on the validation set. Overall, our study highlights the importance of Bangla Complex Named Entity Recognition, particularly in the context of synthetic datasets. Our findings also demonstrate the efficacy of Deep Learning models such as BanglaBERT for NER in Bangla language.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
