HPS:一种层次化的波斯语词干提取方法
HPS: a hierarchical Persian stemming method
摘要:
本文提出了一种新的基于句子中单词词性的层次化波斯语词干提取方法。实现的词干分析器包括哈希表和几个确定性有限自动机在其不同层次的层次结构,以消除前缀和后缀的话。在我们的方法中使用哈希表有两个目的。第一个问题是DFA不支持某些特殊的字,所以哈希表可以部分解决这个问题。第二个目标是通过省略确定性有限自动机所需的时间来加快实现的词干分析器。由于采用了层次化的组织结构,该方法具有快速、灵活的特点。我们在Hamshahri集合和安全新闻(istna.ir)的测试集上的实验表明,我们的方法具有95.37%的平均准确率,甚至在使用具有常见主题的测试集上使用该方法时也有所提高。
英文摘要:
In this paper, a novel hierarchical Persian stemming approach based on the Part-Of-Speech of the word in a sentence is presented. The implemented stemmer includes hash tables and several deterministic finite automata in its different levels of hierarchy for removing the prefixes and suffixes of the words. We had two intentions in using hash tables in our method. The first one is that the DFA don't support some special words, so hash table can partly solve the addressed problem. the second goal is to speed up the implemented stemmer with omitting the time that deterministic finite automata need. Because of the hierarchical organization, this method is fast and flexible enough. Our experiments on test sets from Hamshahri collection and security news (istna.ir) show that our method has the average accuracy of 95.37% which is even improved in using the method on a test set with common topics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。