The ability to learn from limited data, or few-shot learning, is a desirable and often critical requirement for NLP systems. While many existing methods do poorly at learning from a handful of examples, large pretrained language models have recently been shown to be efficient few-shot learners. One approach to few-shot learning, which does not require finetuning of model parameters, is to augment the language model's input with priming text which is typically constructed using task specific descriptions and examples. In this work, we further explore priming-based few-shot learning, with focus on using examples as prompts. We show that presenting examples in the right order is key for generalization. We introduce PERO (Prompting with Examples in the Right Order), where we formulate few-shot learning as search over the set of permutations of the training examples. We show that PERO can learn to generalize efficiently using as few as 10 examples, in contrast to existing approaches. While the newline token is a natural choice for separating the examples in the prompt, we show that learning a new separator token can potentially provide further gains in performance. We demonstrate the effectiveness of the proposed method on the tasks of sentiment classification, natural language inference and fact retrieval. Finally, we analyze the learned prompts to reveal novel insights, including the idea that two training examples in the right order alone can provide competitive performance for sentiment classification and natural language inference.
Zero-shot Entity and Tweet Characterization with Designed Conditional Prompts and Contexts
International Institute of Information Technology, Bangalore(班加罗尔国际信息技术学院);Indraprastha Institute of Information Technology, Delhi(德里印德拉普拉斯塔信息技术学院);International Institute of Information Technology, Hyderabad(海得拉巴国际信息技术学院)
WisPerMed at "Discharge Me!": Advancing Text Generation in Healthcare with Large Language Models, Dynamic Expert Selection, and Priming Techniques on MIMIC-IV
University of Applied Sciences and Arts Dortmund(多特蒙德应用科学与艺术大学);University Hospital Essen(埃森大学医院);Bielefeld University of Applied Sciences and Arts(比勒费尔德应用科学与艺术大学)
Evaluating and Improving Graph to Text Generation with Large Language Models
School of Informatics, University of Edinburgh(爱丁堡大学信息学院);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);School of Computer Science and Informatics, Cardiff University(卡迪夫大学计算机科学与信息学院)
Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems
Center for Artificial Intelligence Research (CAiRE)(人工智能研究中心);The Hong Kong University of Science and Technology(香港科技大学)
面向任务的对话系统使用四个连接的模块,即,自然语言理解(NLU)、对话状态跟踪(DST)、对话策略(DP)和自然语言生成(NLG)。研究的挑战是用最少的样本学习每个模块(即,考虑到与数据汇集相关的高成本,解决这个问题最常见和最有效的技术是迁移学习,其中大语言模型,无论是在文本或特定于任务的数据上进行预训练,都是在少数样本上进行微调。这些方法需要对每个任务进行微调步骤和一组参数。因此,语言模型,例如GPT-2(Radford et al.,2019)和GPT-3(Brown等人,2020),允许通过用很少的例子启动模型来进行少样本学习。在本文中,我们评估了语言模型在NLU,DST,DP和NLG任务中的启动Few-Shot能力。重要的是,我们强调了这种方法目前的局限性,我们讨论了未来工作的可能影响。
How Order-Sensitive Are LLMs? OrderProbe for Deterministic Structural Reconstruction
Peking University(北京大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));City University of Hong Kong(香港城市大学);Fudan University(复旦大学);The Hong Kong Polytechnic University(香港理工大学);Tsinghua University(清华大学);Zhejiang University(浙江大学);University of Illinois Urbana-Champaign(伊利诺伊大学香槟分校);Marquette University(马凯特大学);Juniata College(朱尼阿特学院)