发表机构
Komorebi AI Technologies（Komorebi人工智能技术公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出混合嵌套框架解耦LLM驱动优化的结构与参数，外层LLM提结构草图、内层优化器调参，在三类任务中性能优于纯LLM及纯数值优化基线。
AI 中文摘要
在由语言模型驱动的进化算法中，大语言模型（LLM）是同时更新结构组件（如控制流）和连续参数的单一算子。虽然LLM在结构更新方面表现出色，但在参数优化方面效率低下，会在试错循环中因离散跳转浪费token。为解决这一问题，我们提出了混合嵌套搜索框架：外层循环由LLM提出带有数值缺口的结构草图，内层数值优化器对该草图进行调优。外层和内层求解器均可插拔：任何基于文本的优化器都可与零阶优化器（CMA-ES）、基于梯度的例程或马尔可夫链蒙特卡洛（MCMC）采样器组合。我们在三个科学领域验证了该框架：（i）闭式测试函数上的元优化器；（ii）系统研究与社会困境的基于代码的策略；（iii）近似贝叶斯推理任务。在所有三个领域中，该混合优化器均优于纯LLM驱动搜索和纯数值优化基线。代码见：this https URL
英文摘要
In evolutionary algorithms powered by language models, the LLM acts as a single operator that simultaneously updates structural components (like control flow) and continuous parameters. While LLMs can be good at the first, they are not efficient at the second, wasting tokens taking discrete jumps inside a trial and error loop. We resolve this by formalizing a hybrid nested search, in which an outer loop has the LLM propose a structural sketch, with numeric gaps, and an inner numerical optimizer tunes the sketch. Both the outer and inner solvers are pluggable: any text-based optimizer can be combined with a zero-order optimizer (CMA-ES), gradient-based routines, or MCMC samplers. We validate our framework across three scientific domains: (i) meta-optimizers on closed-form test functions, (ii) code-based policies for systems research and social dilemmas; and (iii) approximate Bayesian inference tasks. Across all three, the hybrid optimizer is superior to both vanilla LLM-driven search and pure numerical optimization baselines. Code at: https://github.com/vicgalle/hybrid-nested-search
CommentsPublished as a conference paper at LM4Sci Workshop @ COLM 2026
