arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.09723stat.MLcs.AIcs.CLcs.LG
收藏

从最佳臂识别视角看高效提示优化

Efficient Prompt Optimization Through the Lens of Best Arm Identification

University of Virginia(弗吉尼亚大学);The Pennsylvania State University(宾夕法尼亚州立大学)

Chengshuai Shi, Kun Yang, Zihan Chen, Jundong Li, Jing Yang, Cong Shen

AI总结:

本文提出TRIPLE框架,通过将提示优化与多臂老虎机中固定预算最佳臂识别相联系,在明确预算约束下高效执行提示选择,并在多任务和多种LLM上显著优于基线,同时扩展至少样本提示示例选择。

摘要:

大型语言模型(LLMs)卓越的指令遵循能力引发了人们对自动寻找优质提示(即提示优化)的日益浓厚的兴趣。现有大多数工作遵循从预生成的候选提示池中进行选择的方案。然而,这些设计主要关注生成策略,而对选择方法的关注有限。尤其是,选择过程中产生的成本(例如访问LLM和评估响应)很少被明确考虑。为了克服这一局限性,本工作提供了一个原则性框架TRIPLE,以在明确的预算约束下高效执行提示选择。TRIPLE建立在提示优化与多臂老虎机(MAB)中固定预算最佳臂识别(BAI-FB)之间建立的新颖联系之上;因此,它能够系统地利用来自BAI-FB的丰富工具箱,并整合提示优化的独特特征。在使用各种LLM的多个广泛采用的任务上进行的广泛实验表明,TRIPLE在满足有限预算约束的同时,相比基线表现出显著的性能提升。作为扩展,提出了TRIPLE的变体以高效地为少样本提示选择示例,也取得了优越的实证性能。

英文摘要:

The remarkable instruction-following capability of large language models (LLMs) has sparked a growing interest in automatically finding good prompts, i.e., prompt optimization. Most existing works follow the scheme of selecting from a pre-generated pool of candidate prompts. However, these designs mainly focus on the generation strategy, while limited attention has been paid to the selection method. Especially, the cost incurred during the selection (e.g., accessing LLM and evaluating the responses) is rarely explicitly considered. To overcome this limitation, this work provides a principled framework, TRIPLE, to efficiently perform prompt selection under an explicit budget constraint. TRIPLE is built on a novel connection established between prompt optimization and fixed-budget best arm identification (BAI-FB) in multi-armed bandits (MAB); thus, it is capable of leveraging the rich toolbox from BAI-FB systematically and also incorporating unique characteristics of prompt optimization. Extensive experiments on multiple well-adopted tasks using various LLMs demonstrate the remarkable performance improvement of TRIPLE over baselines while satisfying the limited budget constraints. As an extension, variants of TRIPLE are proposed to efficiently select examples for few-shot prompts, also achieving superior empirical performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑