arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.02304cs.CLcs.AIcs.LGstat.ML
收藏

Self-Taught Optimizer (STOP):递归自改进代码生成

Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation

Stanford University(斯坦福大学);Microsoft Research(微软研究院);OpenAI

Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Tauman Kalai

AI总结:

本研究提出Self-Taught Optimizer (STOP)框架,以GPT-4为基础构建种子改进器实现脚手架程序的递归自改进,验证了现有LM可编写自调用改进的代码,同时评估了相关安全风险。

摘要:

近期AI系统的多项进展通过提供“脚手架”程序来解决问题,这类程序会构建多次语言模型(LM)调用以生成更优输出,脚手架程序用Python等编程语言编写。本研究中,我们使用融合语言模型的脚手架程序实现自我改进。我们从一个种子“改进器”出发,它通过多次查询LM并返回最优解,依据给定效用函数改进输入程序。随后我们运行该种子改进器以实现其自身的改进。在少量下游任务中,得到的改进后改进器生成的程序性能显著优于种子改进器。语言模型提出了多种自改进策略,包括beam search(束搜索)、genetic algorithms(遗传算法)和simulated annealing(模拟退火)。由于语言模型本身未被修改,这并非完全的递归自改进。尽管如此,该研究证明现代语言模型——我们实验中使用的GPT-4——能够编写可调用自身以实现自我改进的代码。我们考量了自改进技术发展相关的风险,并评估了生成代码绕过沙箱的频率。

英文摘要:

Several recent advances in AI systems solve problems by providing a "scaffolding" program that structures multiple calls to language models (LMs) to generate better outputs. A scaffolding program is written in a programming language such as Python. In this work, we use a language-model-infused scaffolding program to improve itself. We start with a seed "improver" that improves an input program according to a given utility function by querying an LM several times and returning the best solution. We then run this seed improver to improve itself. Across a small set of downstream tasks, the resulting improved improver generates programs with significantly better performance than its seed improver. A variety of self-improvement strategies are proposed by the language model, including beam search, genetic algorithms, and simulated annealing. Since the language models themselves are not altered, this is not full recursive self-improvement. Nonetheless, it demonstrates that a modern language model, GPT-4 in our experiments, is capable of writing code that can call itself to improve itself. We consider concerns around the development of self-improving technologies and evaluate the frequency with which the generated code bypasses a sandbox.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑