arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.00152cs.CL
收藏

学习重写提示以实现个性化文本生成

Learning to Rewrite Prompts for Personalized Text Generation

Google Research(谷歌研究院);University of Michigan(密歇根大学)

Cheng Li, Mingyang Zhang, Qiaozhu Mei, Weize Kong, Michael Bendersky

AI总结:

针对冻结且仅通过API访问的大语言模型,提出一种结合监督学习和强化学习的提示重写方法,自动优化个性化文本生成提示,在三个领域数据集上优于原始及单一学习优化的提示。

摘要:

在大语言模型(LLMs)的推动下,个性化文本生成已成为一个快速发展的研究方向。现有的大多数研究侧重于为特定领域设计专门的模型,或者需要对大语言模型进行微调以生成个性化文本。我们考虑一个典型场景:生成个性化输出的大语言模型是冻结的,且只能通过API访问。在这种约束下,唯一能做的就是改进发送给大语言模型的输入文本(即文本提示),这一过程通常由人工完成。在本文中,我们提出了一种新方法来自动修改用于个性化文本生成的提示。所提出的方法采用由最先进的多阶段个性化生成框架生成的初始提示,并重写其中几个关键组件,这些组件用于总结和综合个人上下文。提示重写器采用了一种将监督学习(SL)和强化学习(RL)串联起来的训练范式,其中SL缩小了RL的搜索空间,而RL则促进了重写器的端到端训练。使用来自三个代表性领域的数据集,我们证明了重写后的提示在性能上优于原始提示以及仅通过监督学习或强化学习优化的提示。对重写后提示的深入分析表明,它们不仅具有人类可读性,而且在资源有限无法采用强化学习训练提示重写器,或部署自动提示重写器进行推理成本高昂时,还能指导人工修改提示。

英文摘要:

Facilitated by large language models (LLMs), personalized text generation has become a rapidly growing research direction. Most existing studies focus on designing specialized models for a particular domain, or they require fine-tuning the LLMs to generate personalized text. We consider a typical scenario in which the large language model, which generates personalized output, is frozen and can only be accessed through APIs. Under this constraint, all one can do is to improve the input text (i.e., text prompts) sent to the LLM, a procedure that is usually done manually. In this paper, we propose a novel method to automatically revise prompts for personalized text generation. The proposed method takes the initial prompts generated by a state-of-the-art, multistage framework for personalized generation and rewrites a few critical components that summarize and synthesize the personal context. The prompt rewriter employs a training paradigm that chains together supervised learning (SL) and reinforcement learning (RL), where SL reduces the search space of RL and RL facilitates end-to-end training of the rewriter. Using datasets from three representative domains, we demonstrate that the rewritten prompts outperform both the original prompts and the prompts optimized via supervised learning or reinforcement learning alone. In-depth analysis of the rewritten prompts shows that they are not only human readable, but also able to guide manual revision of prompts when there is limited resource to employ reinforcement learning to train the prompt rewriter, or when it is costly to deploy an automatic prompt rewriter for inference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑