Izindaba-Tindzaba:面向isiZulu语和Siswati语长短文本的机器学习新闻分类
Izindaba-Tindzaba: Machine learning news categorisation for Long and Short Text for isiZulu and Siswati
AI总结:
针对南非isiZulu语和Siswati语这两种低资源语言,研究构建了新闻主题分类标注数据集,通过数据增强和过采样解决数据短缺与类别不平衡问题,测试多模型与词嵌入组合,发现Word2vec训练的XGBoost、逻辑回归和LSTM表现更优。
摘要:
南非本土/本地语言被归类为低资源语言。因此,为这些语言构建资源,使其能受益于自然语言处理领域的进展至关重要。本研究聚焦于为isiZulu语和Siswati语这两种本土语言创建基于新闻主题分类任务的标注新闻数据集,并展示这些基线分类模型的研究结果。由于这些南非本土语言的数据短缺,研究对创建的数据集进行了增强和过采样,以扩大数据规模并克服类别分类不平衡问题。研究共使用了四种不同的分类模型,分别是Logistic regression(逻辑回归)、Naive bayes(朴素贝叶斯)、XGBoost和LSTM。这些模型在三种不同的词嵌入上进行训练,分别是Bag-Of-Words(词袋模型)、TFIDF和Word2vec。研究结果表明,基于Word2vec训练的XGBoost、Logistic Regression和LSTM的表现优于其他组合。
英文摘要:
Local/Native South African languages are classified as low-resource languages. As such, it is essential to build the resources for these languages so that they can benefit from advances in the field of natural language processing. In this work, the focus was to create annotated news datasets for the isiZulu and Siswati native languages based on news topic classification tasks and present the findings from these baseline classification models. Due to the shortage of data for these native South African languages, the datasets that were created were augmented and oversampled to increase data size and overcome class classification imbalance. In total, four different classification models were used namely Logistic regression, Naive bayes, XGBoost and LSTM. These models were trained on three different word embeddings namely Bag-Of-Words, TFIDF and Word2vec. The results of this study showed that XGBoost, Logistic Regression and LSTM, trained from Word2vec performed better than the other combinations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。