WAZOBIA命名实体识别系统的开发
Development of a WAZOBIA-Named Entity Recognition System
AI总结：
本文面向豪萨语、约鲁巴语和伊博语构建WAZOBIA-NER系统，结合CRF、BiLSTM、BERT与RNN并支持OCR图像输入，在三类实体识别中取得高精确率、召回率、F1和准确率。
摘要：
命名实体识别（Named Entity Recognition，NER）对于信息抽取、机器翻译和情感分析等各种自然语言处理应用至关重要。尽管计算语言学领域对非洲语言的兴趣持续增长，但现有NER系统主要关注英语、欧洲语言以及少数其他全球语言，导致资源匮乏语言存在显著空白。本研究介绍了为尼日利亚三种最重要语言——豪萨语、约鲁巴语和伊博语——定制的WAZOBIA-NER系统的开发。研究首先为每种语言全面汇编标注数据集，以应对数据稀缺和语言多样性挑战。研究探索了最先进的机器学习技术条件随机场（Conditional Random Fields，CRF）以及双向长短期记忆网络（Bidirectional Long Short-Term Memory，BiLSTM）、Transformer双向编码器表示（Bidirectional Encoder Representation from Transformers，Bert）等深度学习模型，并用循环神经网络（Recurrent Neural Network，RNN）进行微调，评估这些方法在识别人物、组织和地点三类实体方面的有效性。该系统利用光学字符识别（optical character recognition，OCR）技术将文本图像转换为机器可读文本，从而使Wazobia系统能够同时接收输入文本和文本图像以进行抽取。系统取得了0.9511的精确率、0.9400的召回率、0.9564的F1分数和0.9301的准确率。模型评估在三种语言上进行，并以精确率、召回率、F1分数和准确率作为关键评估指标。WAZOBIA-NER系统表明，利用当前NLP框架和迁移学习为资源匮乏的非洲语言构建稳健的NER工具是可行的。
英文摘要：
Named Entity Recognition NER is very crucial for various natural language processing applications, including information extraction, machine translation, and sentiment analysis. Despite the ever-increasing interest in African languages within computational linguistics, existing NER systems focus mainly on English, European, and a few other global languages, leaving a significant gap for under-resourced languages. This research presents the development of a WAZOBIA-NER system tailored for the three most prominent Nigerian languages: Hausa, Yoruba, and Igbo. This research begins with a comprehensive compilation of annotated datasets for each language, addressing data scarcity and linguistic diversity challenges. Exploring the state-of-the-art machine learning technique, Conditional Random Fields (CRF) and deep learning models such as Bidirectional Long Short-Term Memory (BiLSTM), Bidirectional Encoder Representation from Transformers (Bert) and fine-tune with a Recurrent Neural Network (RNN), the study evaluates the effectiveness of these approaches in recognizing three entities: persons, organizations, and locations. The system utilizes optical character recognition (OCR) technology to convert textual images into machine-readable text, thereby enabling the Wazobia system to accept both input text and textual images for extraction purposes. The system achieved a performance of 0.9511 in precision, 0.9400 in recall, 0.9564 in F1-score, and 0.9301 in accuracy. The model's evaluation was conducted across three languages, with precision, recall, F1-score, and accuracy as key assessment metrics. The Wazobia-NER system demonstrates that it is feasible to build robust NER tools for under-resourced African languages using current NLP frameworks and transfer learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
