arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.07465cs.CLcs.AI
收藏

基于情景记忆的大语言模型提示方法

Large Language Models Prompting With Episodic Memory

Deakin University(迪肯大学);Applied AI Institute(应用人工智能研究院);ServiceNow Research(ServiceNow研究院)

Dai Do, Quan Tran, Svetha Venkatesh, Hung Le

AI总结:

本研究提出基于情景记忆的提示优化技术POEM,将提示优化转化为强化学习问题,通过记忆存储训练样本与奖励,测试时选高奖励示例序列,在文本分类等任务中性能优于现有方法。

摘要:

提示优化对于提升大语言模型(LLMs)在各类自然语言处理(NLP)任务中的性能至关重要,尤其是在少样本学习场景下——这类场景中训练样本会被直接整合到提示中。尽管利用少样本示例优化提示的研究日益受到关注,但现有提示优化方法往往资源消耗大,或性能表现不佳。本研究提出了一种新颖的提示优化技术——基于情景记忆的提示方法(PrOmpting with Episodic Memory, POEM),该方法简单高效,且具备强大的泛化能力。我们将提示优化视为一个强化学习(Reinforcement Learning, RL)问题,利用情景记忆存储输入数据组合、少样本示例的排列顺序,以及训练过程中观测到的奖励。在测试阶段,我们通过从情景记忆中相似度最高的前k个训练示例里选出总奖励最高的示例序列,为每个测试查询优化示例排列顺序。实验结果表明,在各类文本分类任务中,POEM的性能比TEMPERA、RLPrompt等近期技术高出5.3%以上。此外,我们的方法能很好地适配更广泛的语言理解任务,性能始终优于传统的示例排序启发式方法。

英文摘要:

Prompt optimization is essential for enhancing the performance of Large Language Models (LLMs) in a range of Natural Language Processing (NLP) tasks, particularly in scenarios of few-shot learning where training examples are incorporated directly into the prompt. Despite the growing interest in optimizing prompts with few-shot examples, existing methods for prompt optimization are often resource-intensive or perform inadequately. In this work, we propose PrOmpting with Episodic Memory (POEM), a novel prompt optimization technique that is simple, efficient, and demonstrates strong generalization capabilities. We approach prompt optimization as a Reinforcement Learning (RL) challenge, using episodic memory to archive combinations of input data, permutations of few-shot examples, and the rewards observed during training. In the testing phase, we optimize the sequence of examples for each test query by selecting the sequence that yields the highest total rewards from the top-k most similar training examples in the episodic memory. Our results show that POEM outperforms recent techniques like TEMPERA and RLPrompt by over 5.3% in various text classification tasks. Furthermore, our approach adapts well to broader language understanding tasks, consistently outperforming conventional heuristic methods for ordering examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑