arXiv 2109.086482021-09-09cs.CL

提高阿拉伯语学习者文献易读性的有效测量

Efficient Measuring of Readability to Improve Documents Accessibility for Arabic Language Learners

University of Ferhat Abbas Sétif 1（费尔哈特·阿巴斯·塞蒂夫第一大学）

Sadik Bessou, Ghozlane Chenni

备注：: 8 pages. arXiv admin note: text overlap with arXiv:1911.03776

摘要：

本文提出了一种基于监督的机器学习方法来建立一个分类器，可以识别文本的复杂性，以目前阿拉伯语学习者的文本适合他们的水平。该方法基于机器学习分类方法来区分阅读和理解文本的不同难度。在从在线阿拉伯语网站挖掘的大型语料库上训练了几个模型，并进行了手动注释。该模型使用计数和TF-IDF表示，并应用五种机器学习算法：多项式朴素贝叶斯，伯努利朴素贝叶斯，逻辑回归，支持向量机和随机森林，使用一元和二元特征。为了提取文本的复杂性，通常将层次识别作为一个分类任务来解决。实验结果表明，n-gram特征可以指示文本的阅读水平，并可以大大提高性能，并表明SVM和多项式朴素贝叶斯在预测复杂度方面是最准确的。最好的结果是使用TF-IDF向量，通过基于单词的unigram和bigram的组合进行训练，在四个复杂度级别上的总体准确率为87.14%。

英文摘要：

This paper presents an approach based on supervised machine learning methods to build a classifier that can identify text complexity in order to present Arabic language learners with texts suitable to their levels. The approach is based on machine learning classification methods to discriminate between the different levels of difficulty in reading and understanding a text. Several models were trained on a large corpus mined from online Arabic websites and manually annotated. The model uses both Count and TF-IDF representations and applies five machine learning algorithms; Multinomial Naive Bayes, Bernoulli Naive Bayes, Logistic Regression, Support Vector Machine and Random Forest, using unigrams and bigrams features. With the goal of extracting the text complexity, the problem is usually addressed by formulating the level identification as a classification task. Experimental results showed that n-gram features could be indicative of the reading level of a text and could substantially improve performance, and showed that SVM and Multinomial Naive Bayes are the most accurate in predicting the complexity level. Best results were achieved using TF-IDF Vectors trained by a combination of word-based unigrams and bigrams with an overall accuracy of 87.14% over four classes of complexity.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文