arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1901.04216cs.IRcs.CL
收藏

文本文档中的阿尔巴尼亚语识别

Albanian Language Identification in Text Documents

University of Tirana(地拉那大学)

Klesti Hoxha, Artur Baxhaku

摘要:

在这项工作中,我们调查的准确性标准和国家的最先进的语言识别方法在确定阿尔巴尼亚语的书面文本文件。为此目的,建立了一个由阿尔巴尼亚语新闻文章组成的数据集。我们注意到,当使用遗漏阿尔巴尼亚字母表字母““和”“的测试文档时,准确率会大幅下降,并创建了一个自定义训练语料库,通过实现超过99%的准确率来解决这个问题。根据我们的实验,阿尔巴尼亚语最有效的语言识别方法使用朴素贝叶斯分类器和基于n-gram的分类特征。

英文摘要:

In this work we investigate the accuracy of standard and state-of-the-art language identification methods in identifying Albanian in written text documents. A dataset consisting of news articles written in Albanian has been constructed for this purpose. We noticed a considerable decrease of accuracy when using test documents that miss the Albanian alphabet letters " Ë " and " Ç " and created a custom training corpus that solved this problem by achieving an accuracy of more than 99%. Based on our experiments, the most performing language identification methods for Albanian use a naïve Bayes classifier and n-gram based classification features.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑