arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.01751cs.CL
收藏

在基于启动的少样本学习中重新排序示例有助于

Reordering Examples Helps during Priming-based Few-Shot Learning

Indian Institute of Science(印度科学学院)

Sawan Kumar, Partha Talukdar

摘要:

从有限的数据中学习的能力,或少样本学习,是NLP系统的一个理想的,往往是关键的要求。虽然许多现有的方法在从少数例子中学习方面做得很差,但大型预训练语言模型最近被证明是有效的Few-Shot学习器。不需要模型参数的微调的少样本学习的一种方法是用启动文本来增强语言模型的输入,启动文本通常使用任务特定的描述和示例来构造。在这项工作中,我们进一步探索基于启动的少样本学习,重点是使用示例作为提示。我们表明,以正确的顺序提出的例子是推广的关键。我们引入了PERO(按正确顺序排列示例),其中我们将少样本学习公式化为对训练样本的排列集合的搜索。我们表明,PERO可以学习,有效地推广使用少至10个例子,与现有的方法。虽然换行符标记是分隔提示中的示例的自然选择,但我们表明学习新的分隔符标记可能会进一步提高性能。我们证明了所提出的方法的情感分类,自然语言推理和事实检索的任务上的有效性。最后,我们分析了学习的提示,以揭示新的见解,包括两个训练样本在正确的顺序单独可以提供情感分类和自然语言推理的竞争力的性能的想法。

英文摘要:

The ability to learn from limited data, or few-shot learning, is a desirable and often critical requirement for NLP systems. While many existing methods do poorly at learning from a handful of examples, large pretrained language models have recently been shown to be efficient few-shot learners. One approach to few-shot learning, which does not require finetuning of model parameters, is to augment the language model's input with priming text which is typically constructed using task specific descriptions and examples. In this work, we further explore priming-based few-shot learning, with focus on using examples as prompts. We show that presenting examples in the right order is key for generalization. We introduce PERO (Prompting with Examples in the Right Order), where we formulate few-shot learning as search over the set of permutations of the training examples. We show that PERO can learn to generalize efficiently using as few as 10 examples, in contrast to existing approaches. While the newline token is a natural choice for separating the examples in the prompt, we show that learning a new separator token can potentially provide further gains in performance. We demonstrate the effectiveness of the proposed method on the tasks of sentiment classification, natural language inference and fact retrieval. Finally, we analyze the learned prompts to reveal novel insights, including the idea that two training examples in the right order alone can provide competitive performance for sentiment classification and natural language inference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑