arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.32026cs.AI
收藏

RSI-Opt:用于模型训练的递归自我改进优化器智能体

RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model Training

Purdue University(普渡大学)

Binghang Lu, Yige Sun, Guang Lin

AI总结:

提出RSI-Opt递归自我改进优化器智能体,通过元智能体演化策略,在多任务上显著提升优化器智能体性能,支持跨模型与PDE任务迁移。

摘要:

传统神经网络训练通常遵循预定义的优化方案,这促使人们开发了一种大型语言模型(LLM)智能体,用于监控训练动态并在训练过程中调整优化器设置,但固定的智能体策略无法在多次运行中积累经验。我们引入RSI-Opt,这是一种即插即用的递归自我改进优化器智能体,它会演化指导该过程的知识:包括一个提示词、一套程序手册、一个经过测试的优化器代码库以及一个决策节奏。一个元智能体会利用早期运行的证据重写该策略,而一个存档会保留已评估的策略以进一步演化。在三个训练任务上经过5代演化后,元智能体开发出了可复用的程序和代码,并开展了自身的CPU代理实验。RSI-Opt显著提升了优化器智能体的性能,在演化套件中,使用Opus 5.5时,代表原始空间的z分数从+10.48提升至+14.33;在使用GPT-5.5的未见过任务上,通过跨模型策略迁移,z分数从-0.36提升至+3.11。在Burgers偏微分方程(PDE)任务上,与固定提示词智能体基线相比,使用Opus 5.5时,它将平均相对L₂误差降低了约2.7倍,使用GPT-6.1时降低了8.8倍,尽管PDE的迁移效果因LLM和任务而异。对照实验表明,演化后的策略将可复用的优化知识与对每个任务的适应性相结合:初始决策在已见过的任务上捕获了大部分观察到的增益,而持续的适应性则改善了未见过任务的结果。这些发现证明,在多次训练运行中积累的程序和可执行工具如何能够提升优化器智能体的性能,并支持向科学机器学习的迁移。

英文摘要:

Conventional neural-network training typically follows a predefined optimization recipe, motivating a LLM agent that monitors training dynamics and adjusts optimizer settings as training progresses, but a fixed agent policy does not accumulate experience across runs. We introduce RSI-Opt, a plug-and-play Recursive Self-Improvement Optimizer Agents which evolves the knowledge that guides this process: a prompt, a playbook of procedures, a library of tested optimizer code, and a decision cadence. A meta-agent rewrites this policy using evidence from earlier runs, and an archive retains evaluated policies for further evolution. Across 5 generations on three training tasks, the meta-agents develop reusable procedures and code and conduct their own CPU proxy experiments. RSI-Opt substantially improves optimizer-agent performance, with representative raw-space summary $z$-scores increasing from $+10.48$ to $+14.33$ on the evolution suite with Opus 5.5 and from $-0.36$ to $+3.11$ on held-out tasks with GPT-5.5 through cross-model policy transfer. On the Burgers PDE, it reduces mean relative $L_2$ error by approximately 2.7-fold with Opus 5.5 and 8.8-fold with GPT-6.1 over the fixed-prompt agent baseline, although PDE transfer varies across LLMs and tasks. Controlled comparisons show that the evolved policy combines reusable optimization knowledge with adaptation to each task: initial decisions capture most observed gains on seen tasks, while continued adaptation improves outcomes on unseen tasks. These findings demonstrate how procedures and executable tools accumulated across training runs can improve optimizer-agent performance and support transfer to scientific machine learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑