arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.04486cs.CLcs.AI
收藏

面向上下文学习的主动样例选择

Active Example Selection for In-Context Learning

University of Chicago(芝加哥大学)

Yiming Zhang, Shi Feng, Chenhao Tan

摘要:

通过少量的演示示例,大规模语言模型显示出强大的能力,可以通过从这些示例中进行上下文学习来执行各种任务,而无需任何微调。我们证明,上下文学习的性能可以是非常不稳定的样本的例子,表明如何语言模型获取信息的特质。我们制定的上下文学习作为一个顺序决策问题的例子选择,并提出了一个强化学习算法,用于确定可推广的政策,选择示范的例子。对于GPT-2,我们学习的策略表现出很强的泛化能力,在训练中看不见的任务,平均$5.8\%$的改善。从我们学习的策略中选择的例子甚至可以在GPT-3 Ada上实现小的改进。然而,在更大的GPT-3模型上,这种改进会减少,这表明大语言模型的新功能。

英文摘要:

With a handful of demonstration examples, large-scale language models show strong capability to perform various tasks by in-context learning from these examples, without any fine-tuning. We demonstrate that in-context learning performance can be highly unstable across samples of examples, indicating the idiosyncrasies of how language models acquire information. We formulate example selection for in-context learning as a sequential decision problem, and propose a reinforcement learning algorithm for identifying generalizable policies to select demonstration examples. For GPT-2, our learned policies demonstrate strong abilities of generalizing to unseen tasks in training, with a $5.8\%$ improvement on average. Examples selected from our learned policies can even achieve a small improvement on GPT-3 Ada. However, the improvement diminishes on larger GPT-3 models, suggesting emerging capabilities of large language models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑