arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节（Veterans Day），12 日无更新
arXiv 2504.16005cs.CLcs.AIcs.NEstat.ML
收藏

CAPO: Cost-Aware Prompt Optimization（成本感知提示优化）

CAPO: Cost-Aware Prompt Optimization

LMU Munich（慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）

Tom Zehle, Moritz Schlager, Timo Heiß, Matthias Feurer

备注：
Submitted to AutoML 2025
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有自动提示优化成本高的问题，提出融合AutoML技术的CAPO进化算法，通过竞速减评估、多目标优化平衡性能与长度，在多实验中多数场景优于现有方法且成本更低。

摘要：

大语言模型（LLM）仅通过提示引导即可解决各类任务，彻底革新了自然语言处理领域，但其性能对提示的表述高度敏感。自动提示优化通过寻找最优提示解决了这一挑战，但现有方法需要大量LLM调用和输入令牌，导致提示优化成本高昂。我们提出CAPO（Cost-Aware Prompt Optimization，成本感知提示优化）算法，该算法融合AutoML（自动化机器学习）技术以提升提示优化效率。CAPO是一种以LLM为算子的进化方法，引入竞速（racing）机制减少评估次数，采用多目标优化平衡性能与提示长度，可联合优化指令和少样本示例，同时利用任务描述提升鲁棒性。我们在多样数据集和LLM上开展了大量实验，结果表明CAPO在15种情况中有11种优于当前最优的离散提示优化方法，准确率提升最高达21个百分点。该算法在更小预算下即可实现更优性能，通过竞速机制减少评估量，并借助长度惩罚缩短平均提示长度，兼具成本效益与成本感知能力。即使没有少样本示例，CAPO的表现仍优于同类方法，且通常对初始提示保持鲁棒性。CAPO通过提升成本效益，为推动提示优化更强大、更易获取迈出了重要一步。

英文摘要：

Large language models (LLMs) have revolutionized natural language processing by solving a wide range of tasks simply guided by a prompt. Yet their performance is highly sensitive to prompt formulation. While automatic prompt optimization addresses this challenge by finding optimal prompts, current methods require a substantial number of LLM calls and input tokens, making prompt optimization expensive. We introduce CAPO (Cost-Aware Prompt Optimization), an algorithm that enhances prompt optimization efficiency by integrating AutoML techniques. CAPO is an evolutionary approach with LLMs as operators, incorporating racing to save evaluations and multi-objective optimization to balance performance with prompt length. It jointly optimizes instructions and few-shot examples while leveraging task descriptions for improved robustness. Our extensive experiments across diverse datasets and LLMs demonstrate that CAPO outperforms state-of-the-art discrete prompt optimization methods in 11/15 cases with improvements up to 21%p in accuracy. Our algorithm achieves better performances already with smaller budgets, saves evaluations through racing, and decreases average prompt length via a length penalty, making it both cost-efficient and cost-aware. Even without few-shot examples, CAPO outperforms its competitors and generally remains robust to initial prompts. CAPO represents an important step toward making prompt optimization more powerful and accessible by improving cost-efficiency.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑