Transformer能通过Wug测试吗？神经形态学拐点模型中的神经元偏差调节
Can a Transformer Pass the Wug Test? Tuning Copying Bias in Neural Morphological Inflection Models
摘要：
深度学习序列模型已成功应用于形态学拐点的任务。SIGMORPHON共享任务在过去几年中的结果表明，这样的模型可以表现良好，但前提是训练数据覆盖大量不同的引理，或者在训练中也可以看到在测试时发生变化的引理，就像这些任务中的情况一样。令人惊讶的是，标准模型，如Transformer，当被要求改变以前看不见的词元时，几乎完全无法概括变化模式-即在“wug测试”-类似的情况下。虽然建立的数据增强技术可以用来减轻这一缺点，通过引入复制偏见幻觉合成新词形式使用的字母在手头的语言，我们表明，为了更有效，幻觉过程需要注意的子串的音节长度，而不是个别字符或茎。我们报告了一个显着的性能改善与我们的基于子串的幻觉模型比以前的数据幻觉方法时，训练和测试数据不重叠，在他们的引理。
英文摘要：
Deep learning sequence models have been successfully applied to the task of morphological inflection. The results of the SIGMORPHON shared tasks in the past several years indicate that such models can perform well, but only if the training data cover a good amount of different lemmata, or if the lemmata that are inflected at test time have also been seen in training, as has indeed been largely the case in these tasks. Surprisingly, standard models such as the Transformer almost completely fail at generalizing inflection patterns when asked to inflect previously unseen lemmata -- i.e. under "wug test"-like circumstances. While established data augmentation techniques can be employed to alleviate this shortcoming by introducing a copying bias through hallucinating synthetic new word forms using the alphabet in the language at hand, we show that, to be more effective, the hallucination process needs to pay attention to substrings of syllable-like length rather than individual characters or stems. We report a significant performance improvement with our substring-based hallucination model over previous data hallucination methods when training and test data do not overlap in their lemmata.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
