arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.04252cs.CL
收藏

元学习的组成概括

Meta-Learning to Compositionally Generalize

University of Edinburgh(爱丁堡大学);University of Amsterdam(阿姆斯特丹大学)

Henry Conklin, Bailin Wang, Kenny Smith, Ivan Titov

摘要:

自然语言是合成的;句子的意义是其各部分意义的函数。这种特性使人类能够创造和解释新的句子,在他们先前的经验之外进行有力的概括。神经网络已经被证明难以处理这种泛化,特别是在设计用于评估组合泛化的任务上表现不佳(即训练和测试分布的不同方式对于组合策略来说是微不足道的)。他们在这些任务上的糟糕表现可能部分是由于监督学习的性质,它假设训练和测试数据来自同一分布。我们实现了一个元学习增强版本的监督学习,其目标直接优化分布外泛化。我们通过对现有的训练数据进行子采样来构建元学习的任务对。每一对任务都被构造成包含由相似性度量确定的相关示例,以防止模型记住它们的输入。在COGS和SCAN数据集上的实验结果表明,我们的相似性驱动的元学习可以提高泛化性能。

英文摘要:

Natural language is compositional; the meaning of a sentence is a function of the meaning of its parts. This property allows humans to create and interpret novel sentences, generalizing robustly outside their prior experience. Neural networks have been shown to struggle with this kind of generalization, in particular performing poorly on tasks designed to assess compositional generalization (i.e. where training and testing distributions differ in ways that would be trivial for a compositional strategy to resolve). Their poor performance on these tasks may in part be due to the nature of supervised learning which assumes training and testing data to be drawn from the same distribution. We implement a meta-learning augmented version of supervised learning whose objective directly optimizes for out-of-distribution generalization. We construct pairs of tasks for meta-learning by sub-sampling existing training data. Each pair of tasks is constructed to contain relevant examples, as determined by a similarity metric, in an effort to inhibit models from memorizing their input. Experimental results on the COGS and SCAN datasets show that our similarity-driven meta-learning can improve generalization performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑