评估神经机器翻译中的结构泛化
Evaluating Structural Generalization in Neural Machine Translation
AI总结：
针对机器翻译中结构泛化评估不足的问题，构建了受控数据集SGET，评估发现神经机器翻译模型在结构泛化上比词汇泛化更困难，且语义解析与机器翻译性能趋势不同，凸显跨任务评估的重要性。
摘要：
组合泛化是指泛化到先前观察到的单词和句法结构的新组合的能力。由于它被认为是神经模型的一种理想属性，最近的工作也评估了机器翻译以及语义解析中的组合泛化。然而，以前对机器翻译的评估主要集中在词汇泛化（即泛化到已知单词的未见组合）上。因此，模型在多大程度上能够翻译需要结构泛化（即泛化到不同类型的句法结构）的句子仍不清楚。为了解决这个问题，我们构建了SGET，这是一个机器翻译数据集，涵盖了各种类型的组合泛化，并对单词和句子结构进行了控制。我们在SGET上评估神经机器翻译模型，并表明它们在结构泛化方面比在词汇泛化方面更困难。我们还发现了语义解析和机器翻译中不同的性能趋势，这表明了跨各种任务进行评估的重要性。
英文摘要：
Compositional generalization refers to the ability to generalize to novel combinations of previously observed words and syntactic structures. Since it is regarded as a desired property of neural models, recent work has assessed compositional generalization in machine translation as well as semantic parsing. However, previous evaluations with machine translation have focused mostly on lexical generalization (i.e., generalization to unseen combinations of known words). Thus, it remains unclear to what extent models can translate sentences that require structural generalization (i.e., generalization to different sorts of syntactic structures). To address this question, we construct SGET, a machine translation dataset covering various types of compositional generalization with control of words and sentence structures. We evaluate neural machine translation models on SGET and show that they struggle more in structural generalization than in lexical generalization. We also find different performance trends in semantic parsing and machine translation, which indicates the importance of evaluations across various tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
