通用Lemmatizer：一个通用同源树库的Sequence to Sequence Lemmatizer模型
Universal Lemmatizer: A Sequence to Sequence Model for Lemmatizing Universal Dependencies Treebanks
摘要：
在本文中，我们提出了一种新的基于序列到序列的神经网络架构和形态句法上下文表示的词形化方法。在所提出的方法中，我们的上下文敏感的lemmatizer产生的引理一个字符的基础上的表面形式的字符和它的形态句法特征从形态标记的时间。我们认为，一个滑动窗口上下文表示遭受稀疏，而在大多数情况下，一个词的形态句法特征带来足够的信息来解决词元歧义，同时保持上下文表示密集和更实用的机器学习系统。此外，我们研究了两种不同的数据增强方法，利用自动编码器训练和形态换能器，特别是有利于低资源的语言。我们在52种不同的语言和76种不同的树库上评估了我们的lemmatizer，表明我们的系统优于所有最新的基线系统。与最好的整体基线UDPipe Future相比，我们的系统在76个树库中的62个上的表现优于它，平均相对减少了19%的错误。lemmatizer和所有训练过的模型都可以作为Apache 2.0许可证下Turku神经解析管道的一部分。
英文摘要：
In this paper we present a novel lemmatization method based on a sequence-to-sequence neural network architecture and morphosyntactic context representation. In the proposed method, our context-sensitive lemmatizer generates the lemma one character at a time based on the surface form characters and its morphosyntactic features obtained from a morphological tagger. We argue that a sliding window context representation suffers from sparseness, while in majority of cases the morphosyntactic features of a word bring enough information to resolve lemma ambiguities while keeping the context representation dense and more practical for machine learning systems. Additionally, we study two different data augmentation methods utilizing autoencoder training and morphological transducers especially beneficial for low resource languages. We evaluate our lemmatizer on 52 different languages and 76 different treebanks, showing that our system outperforms all latest baseline systems. Compared to the best overall baseline, UDPipe Future, our system outperforms it on 62 out of 76 treebanks reducing errors on average by 19% relative. The lemmatizer together with all trained models is made available as a part of the Turku-neural-parsing-pipeline under the Apache 2.0 license.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
