arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.11486cs.CL
收藏

MetaPrompting:学会更好地学习

MetaPrompting: Learning to Learn Better Prompts

Harbin Institute of Technology(哈尔滨工业大学)

Yutai Hou, Hongyuan Dong, Xinghao Wang, Bohan Li, Wanxiang Che

摘要:

压缩方法被认为是Few-Shot自然语言处理的关键技术之一。最近的研究提示从离散的令牌为基础的“硬提示”,连续的“软提示”,采用可学习的向量作为伪提示令牌,并取得更好的性能。虽然这些软激励方法显示出良好的前景,但它们在很大程度上依赖于良好的初始化。不幸的是,获得软提示的完美初始化需要理解内部语言模型的工作和精心设计,这不是一件容易的事情,并且必须为每个新任务从头开始。针对这一问题,本文提出了一种通用的软提示方法MetaPropagating,该方法采用模型无关的Meta学习算法,自动寻找更好的提示初始化,以快速适应新的提示任务,实验表明MetaPropagating解决了软提示初始化问题,并在四个不同的数据集上取得了显著的改进(单次设置的精度提高了6个点以上),实现了最先进的性能。

英文摘要:

Prompting method is regarded as one of the crucial progress for few-shot nature language processing. Recent research on prompting moves from discrete tokens based ``hard prompts'' to continuous ``soft prompts'', which employ learnable vectors as pseudo prompt tokens and achieve better performance. Though showing promising prospects, these soft-prompting methods are observed to rely heavily on good initialization to take effect. Unfortunately, obtaining a perfect initialization for soft prompts requires understanding of inner language models working and elaborate design, which is no easy task and has to restart from scratch for each new task. To remedy this, we propose a generalized soft prompting method called MetaPrompting, which adopts the well-recognized model-agnostic meta-learning algorithm to automatically find better prompt initialization that facilitates fast adaptation to new prompting tasks.Extensive experiments show MetaPrompting tackles soft prompt initialization problem and brings significant improvement on four different datasets (over 6 points improvement in accuracy for 1-shot setting), achieving new state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑