提示优化对于提升大语言模型(LLMs)在各类自然语言处理(NLP)任务中的性能至关重要,尤其是在少样本学习场景下——这类场景中训练样本会被直接整合到提示中。尽管利用少样本示例优化提示的研究日益受到关注,但现有提示优化方法往往资源消耗大,或性能表现不佳。本研究提出了一种新颖的提示优化技术——基于情景记忆的提示方法(PrOmpting with Episodic Memory, POEM),该方法简单高效,且具备强大的泛化能力。我们将提示优化视为一个强化学习(Reinforcement Learning, RL)问题,利用情景记忆存储输入数据组合、少样本示例的排列顺序,以及训练过程中观测到的奖励。在测试阶段,我们通过从情景记忆中相似度最高的前k个训练示例里选出总奖励最高的示例序列,为每个测试查询优化示例排列顺序。实验结果表明,在各类文本分类任务中,POEM的性能比TEMPERA、RLPrompt等近期技术高出5.3%以上。此外,我们的方法能很好地适配更广泛的语言理解任务,性能始终优于传统的示例排序启发式方法。
英文摘要:
Prompt optimization is essential for enhancing the performance of Large Language Models (LLMs) in a range of Natural Language Processing (NLP) tasks, particularly in scenarios of few-shot learning where training examples are incorporated directly into the prompt. Despite the growing interest in optimizing prompts with few-shot examples, existing methods for prompt optimization are often resource-intensive or perform inadequately. In this work, we propose PrOmpting with Episodic Memory (POEM), a novel prompt optimization technique that is simple, efficient, and demonstrates strong generalization capabilities. We approach prompt optimization as a Reinforcement Learning (RL) challenge, using episodic memory to archive combinations of input data, permutations of few-shot examples, and the rewards observed during training. In the testing phase, we optimize the sequence of examples for each test query by selecting the sequence that yields the highest total rewards from the top-k most similar training examples in the episodic memory. Our results show that POEM outperforms recent techniques like TEMPERA and RLPrompt by over 5.3% in various text classification tasks. Furthermore, our approach adapts well to broader language understanding tasks, consistently outperforming conventional heuristic methods for ordering examples.
Echo: A Large Language Model with Temporal Episodic Memory
Institute of AI Education, East China Normal University(华东师范大学人工智能教育学院);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院)
Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks
Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所);Intel Labs(英特尔实验室);University of Texas at Austin(德克萨斯大学奥斯汀分校);Princeton University(普林斯顿大学);Princeton Neuroscience Institute(普林斯顿神经科学研究所);New York University(纽约大学)
CapMem: A Benchmark for Caption-Based Episodic Memory in Egocentric Video
University College London(伦敦大学学院);Imperial College London(帝国理工学院);Durham University(杜伦大学);Shanghai Jiao Tong University(上海交通大学);Queen Mary University of London(伦敦玛丽女王大学)
UniMem: Complementary Episodic-to-Parametric Memory for Boundary-Agnostic Task Streams
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院通用人工智能国家重点实验室);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Meituan(美团);AI Centre, Department of Computer Science, University College London(伦敦大学学院计算机科学系人工智能中心)
Video-EM: Event-Centric Episodic Memory for Long-Form Video Understanding
City University of Hong Kong(香港城市大学);University of Science and Technology of China(中国科学技术大学);Tianjin University(天津大学);Nanjing University(南京大学);Zhejiang University(浙江大学);The Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究所(TeleAI))
GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memory
Integrated Vision and Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)(整合视觉与语言实验室,电气工程学院,韩国科学技术院(KAIST));Visual Intelligence Research Section, Superintelligence Creative Research Laboratory, Electronics and Telecommunications Research Institute (ETRI)(视觉智能研究部,超智能创意研究实验室,电子电信研究院)
SYNAPSE: Empowering LLM Agents with Episodic-Semantic Memory via Spreading Activation
School of Computing, University of Georgia(佐治亚大学计算机学院);Department of Biosystems Engineering and Soil Science, University of Tennessee(田纳西大学生物系统工程与土壤科学系);Department of Biomedical Informatics, University of Colorado School of Medicine(科罗拉多医学院生物医学信息学系);Department of Biomedical Engineering, New Jersey Institute of Technology(新泽西理工学院生物医学工程系);Department of Physics and Astronomy, The University of Georgia(佐治亚大学物理与天文学系)
Episodic Memory Verbalization using Hierarchical Representations of Life-Long Robot Experience
Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology(人机学与机器人研究所,卡尔斯鲁厄理工学院);Computer Science Department, Columbia University(计算机科学系,哥伦比亚大学)
Role-aware Heuristic Episodic Attention for Conversational LLMs
National Key Laboratory of Parallel and Distributed Computing(平行与分布式计算国家重点实验室);College of Computer Science and Technology(计算机科学与技术学院);National University of Defense Technology(国防科学技术大学)
Causal Episodic Memory for Feedback-Driven Agent Repair
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), VNU-HCM(胡志明市理工大学计算机科学与工程学院(HCMUT,VNU-HCM))
Lessons from Neuroscience for AI: How integrating Actions, Compositional Structure and Episodic Memory could enable Safe, Interpretable and Human-Like AI
Role-aware Heuristic Episodic Attention for Conversational LLMs
National Key Laboratory of Parallel and Distributed Computing(并行与分布式计算国家重点实验室);College of Computer Science and Technology(计算机科学与技术学院);National University of Defense Technology(国防科技大学)
Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Institute of Science Tokyo(东京科学研究所);The Hong Kong Polytechnic University(香港理工大学)