使用拐点表和原始文本的上下文敏感神经元Lemmatization的训练数据扩充
Training Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text
摘要：
词形化的目的是通过将词的变形形式与其字典形式相关联来减少稀疏数据问题。使用上下文可以帮助，无论是看不见的和模棱两可的话。然而，大多数上下文敏感的方法需要完整的词元注释的句子进行训练，这在低资源语言中可能是稀缺的或不可用的。此外（如此处所示），在低资源设置中，词形化器可以从$n$标记的不同单词（类型）的示例中学习到比从$n$（连续）标记的标记中更多的信息，因为后者包含的不同类型要少得多。为了将基于类型的学习的效率与上下文的好处相结合，我们提出了一种训练上下文敏感的lemmatizer的方法，该方法使用很少或没有标记的语料库数据，使用UniMorph项目的变形表和维基百科的原始文本示例，为明确的UniMorph示例提供句子上下文。尽管这些都是明确的例子，但该模型成功地从它们中概括出来，与不使用上下文的基线相比，结果得到了改善（总体上，特别是在看不见的单词上）。
英文摘要：
Lemmatization aims to reduce the sparse data problem by relating the inflected forms of a word to its dictionary form. Using context can help, both for unseen and ambiguous words. Yet most context-sensitive approaches require full lemma-annotated sentences for training, which may be scarce or unavailable in low-resource languages. In addition (as shown here), in a low-resource setting, a lemmatizer can learn more from $n$ labeled examples of distinct words (types) than from $n$ (contiguous) labeled tokens, since the latter contain far fewer distinct types. To combine the efficiency of type-based learning with the benefits of context, we propose a way to train a context-sensitive lemmatizer with little or no labeled corpus data, using inflection tables from the UniMorph project and raw text examples from Wikipedia that provide sentence contexts for the unambiguous UniMorph examples. Despite these being unambiguous examples, the model successfully generalizes from them, leading to improved results (both overall, and especially on unseen words) in comparison to a baseline that does not use context.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
