Nguni语言的规范和表面形态分割
Canonical and Surface Morphological Segmentation for Nguni Languages
摘要：
形态分割涉及将单词分解为语素，语素是语言中最小的意义承载单位。这是一个重要的NLP任务，对于形态丰富的黏着语言，如南部非洲Nguni语言群。在本文中，我们调查监督和无监督模型的形态分割的两个变种：规范和表面分割。我们训练用于规范分割的序列到序列模型，其中底层词素可能不等于单词的表面形式，以及用于表面分割的条件随机场（CRF）。Transformers在关注规范分割方面优于LSTM，在4种语言中获得了72.5%的平均F1分数。在表面分割上，基于神经网络的CRF优于双向LSTM-CRF，获得平均97.1%的F1。在无监督设置中，使用字符级LSTM语言模型的基于熵的方法未能优于Morfessor基线，而在某些语言上，这两种方法的性能都没有比随机基线好得多。我们希望监督分割模型的高性能将有助于为Nguni语言开发更好的NLP工具。
英文摘要：
Morphological Segmentation involves decomposing words into morphemes, the smallest meaning-bearing units of language. This is an important NLP task for morphologically-rich agglutinative languages such as the Southern African Nguni language group. In this paper, we investigate supervised and unsupervised models for two variants of morphological segmentation: canonical and surface segmentation. We train sequence-to-sequence models for canonical segmentation, where the underlying morphemes may not be equal to the surface form of the word, and Conditional Random Fields (CRF) for surface segmentation. Transformers outperform LSTMs with attention on canonical segmentation, obtaining an average F1 score of 72.5% across 4 languages. Feature-based CRFs outperform bidirectional LSTM-CRFs to obtain an average of 97.1% F1 on surface segmentation. In the unsupervised setting, an entropy-based approach using a character-level LSTM language model fails to outperforms a Morfessor baseline, while on some of the languages neither approach performs much better than a random baseline. We hope that the high performance of the supervised segmentation models will help to facilitate the development of better NLP tools for Nguni languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
