Kvistur 2.0：冰岛语的BiLSTM复合分路器
Kvistur 2.0: a BiLSTM Compound Splitter for Icelandic
摘要：
在本文中，我们提出了一个基于字符的BiLSTM模型分裂冰岛复合词，并显示了不同数量的训练数据如何影响模型的性能。冰岛的化合物生产率很高，新的化合物不断被创造出来。这会导致大量的词汇表外（OOV）单词，对许多NLP工具的性能产生负面影响。我们的模型是在冰岛形态数据库中的290万个独特单词形式及其组成结构的数据集上训练的。该模型学习如何将复合词分为两部分，并可用于导出任何单词形式的组成结构。知道单词形式的组成结构使得有可能为给定的任务生成最佳分割，例如，一个完整的分裂的子词标记，或在词性标注的情况下，分裂的OOV字，直到最大的已知形态heads.The模型优于其他先前发表的方法进行评估时，对语料库的手动分裂的单词形式。这种方法已被集成到Kvistur，冰岛复合词分析器。
英文摘要：
In this paper, we present a character-based BiLSTM model for splitting Icelandic compound words, and show how varying amounts of training data affects the performance of the model. Compounding is highly productive in Icelandic, and new compounds are constantly being created. This results in a large number of out-of-vocabulary (OOV) words, negatively impacting the performance of many NLP tools. Our model is trained on a dataset of 2.9 million unique word forms and their constituent structures from the Database of Icelandic Morphology. The model learns how to split compound words into two parts and can be used to derive the constituent structure of any word form. Knowing the constituent structure of a word form makes it possible to generate the optimal split for a given task, e.g., a full split for subword tokenization, or, in the case of part-of-speech tagging, splitting an OOV word until the largest known morphological head is found. The model outperforms other previously published methods when evaluated on a corpus of manually split word forms. This method has been integrated into Kvistur, an Icelandic compound word analyzer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
