（不）解决形态学拐点：引理重叠牺牲性地提高模型的性能
(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performance
摘要：
在形态学领域，屈折是一个基本而重要的任务，近年来获得了很大的吸引力，主要是通过SIGMORPHON的共享任务。所有语言的平均准确率都在0.9以上，即使提供的数据很少，该任务也被认为主要是使用相对通用的神经seq 2seq模型解决的。在这项工作中，我们建议重新评估形态拐点模型，采用更硬的训练测试分裂，这将挑战模型的泛化能力。特别是，相对于天真的分裂的形式，我们提出了一个分裂的引理方法来挑战现有的基准性能。我们对SIGMORPHON 2020年共享任务中排名最高的三个系统进行的实验表明，对于所包含的90种语言，lemma-split在宏观平均水平上平均下降了30个百分点。这种影响对于低资源语言最为显著，下降幅度高达95分，但即使是高资源语言平均也会下降约10分。我们的研究结果清楚地表明，将拐点推广到看不见的引理还远远没有解决，这为推广更复杂的模型提供了一种简单而有效的方法。
英文摘要：
In the domain of Morphology, Inflection is a fundamental and important task that gained a lot of traction in recent years, mostly via SIGMORPHON's shared-tasks. With average accuracy above 0.9 over the scores of all languages, the task is considered mostly solved using relatively generic neural seq2seq models, even with little data provided. In this work, we propose to re-evaluate morphological inflection models by employing harder train-test splits that will challenge the generalization capacity of the models. In particular, as opposed to the na{ï}ve split-by-form, we propose a split-by-lemma method to challenge the performance on existing benchmarks. Our experiments with the three top-ranked systems on the SIGMORPHON's 2020 shared-task show that the lemma-split presents an average drop of 30 percentage points in macro-average for the 90 languages included. The effect is most significant for low-resourced languages with a drop as high as 95 points, but even high-resourced languages lose about 10 points on average. Our results clearly show that generalizing inflection to unseen lemmas is far from being solved, presenting a simple yet effective means to promote more sophisticated models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
