arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.16006cs.CL
收藏

基于有限状态机的乌兹别克语规则词形化算法

Development of a rule-based lemmatization algorithm through Finite State Machine for Uzbek language

Urgench State University(乌尔根奇国立大学)

Maksud Sharipov, Ogabek Sobirov

摘要:

词形化是自然语言处理的核心概念之一,因此创建词形化工具是一项重要的任务。本文讨论了乌兹别克语的一个词形化算法的构造。这项工作的主要目的是通过有限状态机删除乌兹别克语中的词缀,并确定该词的词元(可在词典中找到的词)。去除词缀的过程使用词缀和词性知识的数据库。这个词形转换包括乌兹别克语的一般规则和词性数据、词缀、词缀分类、根据每个类别的有限状态机去除词缀以及这个词形转换的定义。

英文摘要:

Lemmatization is one of the core concepts in natural language processing, thus creating a lemmatization tool is an important task. This paper discusses the construction of a lemmatization algorithm for the Uzbek language. The main purpose of the work is to remove affixes of words in the Uzbek language by means of the finite state machine and to identify a lemma (a word that can be found in the dictionary) of the word. The process of removing affixes uses a database of affixes and part of speech knowledge. This lemmatization consists of the general rules and a part of speech data of the Uzbek language, affixes, classification of affixes, removing affixes on the basis of the finite state machine for each class, as well as a definition of this word lemma.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑