arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.11482cs.AIcs.CL
收藏

为AI系统引入元提示

Meta Prompting for AI Systems

Tsinghua University(清华大学);Shanghai Qi Zhi Institute(上海期智研究院)

Yifan Zhang, Yang Yuan, Andrew Chi-Chih Yao

AI总结:

通过元提示框架提升大语言模型的推理能力,实现高效的问题解决与自我优化。

摘要:

我们介绍了元提示(MP),一种通过关注任务的正式结构而非内容特定的示例来提升大语言模型(LLMs)推理能力的框架。我们为这一范式建立了理论基础,将MP形式化为一个函子,将任务的范畴映射到结构化提示的范畴,从而保证组合性问题解决策略可以系统地分解为模块化提示结构。我们扩展这一概念到递归元提示(RMP),一种自动化过程,其中LLM可以生成并优化其自身的提示。我们将这种自我改进循环形式化为一个monad,提供了一个原则性的自动提示工程框架。我们的主张通过广泛的实验得到验证,证明一个Qwen-72B基础模型在单个无示例依赖的元提示指导下,在MATH、GSM8K和24点游戏上取得了最先进的结果。这些结果在传统少样本方法上实现了显著的token效率提升。项目页面:https://github.com/meta-prompting/meta-prompting。

英文摘要:

We introduce Meta Prompting (MP), a framework that emphasizes the formal structure of a task rather than content-specific worked examples. We give a categorical formalization in which a functor maps typed task transformations to typed prompt transformations. Functoriality encodes preservation of identities and composition, but it does not by itself guarantee semantic correctness. We extend MP to Recursive Meta Prompting (RMP), in which a proposer model generates candidate prompt edits, a validator enforces the edit schema, and an executor model uses the resulting prompt. We model the accumulation of valid edit scripts with the Writer monad and their application by a monoid action on prompts. This construction makes accumulated edits independent of parenthesization; it does not guarantee convergence or improved task accuracy. Empirically, a Qwen-72B base model guided by example-free meta-prompts attains 46.3% pass@1 on MATH and 83.5% accuracy on GSM8K. Separately, an MP-CR agent synthesizes a tool-executed Game of 24 solver; the original experiment reports 100% success on 1,362 instances, subject to exact re-verification from the corresponding input and per-instance outputs. The batch amortizes one LLM response across all puzzles, so its token accounting is not protocol-matched to per-instance few-shot or search-based methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑