发表机构
Meta; RWTH Aachen University（Meta; 亚琛工业大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出代理式贝叶斯优化范式，构建Sara代理与lenz后端，在合成及真实基准测试中，其兼具可靠性与性能，还可动态重构优化问题。
AI 中文摘要
贝叶斯优化（BO）已成为样本高效优化的标准工具，其效率源于通用统计先验驱动的不确定性感知搜索。更丰富的领域先验原则上可改进BO，但通过定制核函数或问题结构对其进行编码十分困难，实际中很少实现。大型语言模型（LLM）可通过将自然语言、代码和文档中的非正式先验直接提供给优化器，帮助规避这一难题。然而，现有基于LLM的BO方法要么将LLM置于固定角色（代理、获取代理或配置接口），要么赋予其广泛控制权，牺牲了BO可靠性所依赖的系统探索。我们提出代理式贝叶斯优化：一种范式，其中LLM代理是BO循环的核心决策者，而贝叶斯后端提供不确定性感知的优化基础。该代理可配置问题、查询后端、选择并提交评估，还可在运行过程中调整优化策略，例如收紧边界、切换获取函数、提出针对性评估，甚至根据新指令或观察到的证据重新构建问题。我们将这一想法实例化为Sara，一个代理增强的自动研究代理，以及lenz，一个模块化的BoTorch后端，代理可通过结构化接口对其进行检查和修改。在合成及真实世界基准测试中，Sara在无先验知识的情况下保持了最先进BO的可靠性，优于基于LLM的基线，且利用自然语言先验实现了超出标准BO的性能。我们进一步在动态场景中展示了代理式BO的实用价值，其中Sara可在需求变化时即时重新配置整个优化问题，这是标准BO此前不具备的能力。
英文摘要
Bayesian optimization (BO) has become the standard tool for sample-efficient optimization and owes its efficiency to uncertainty-aware search driven by generic statistical priors. Richer domain priors can improve BO in principle, but encoding them through tailored kernels or problem structure is difficult and rarely done in practice. LLMs can help sidestep this difficulty by making informal priors from natural language, code, and documentation directly available to the optimizer. However, existing LLM-based BO methods either insert the LLM into a fixed role (surrogate, acquisition proxy, or configuration interface) or hand it broad control, sacrificing the systematic exploration that makes BO reliable. We introduce agentic Bayesian optimization: a paradigm in which an LLM agent is the central decision maker in the BO loop while a Bayesian backend provides the uncertainty-aware optimization substrate. The agent configures the problem, queries the backend, selects and commits evaluations, and can revise the optimization strategy during the run by tightening bounds, switching acquisition functions, proposing targeted evaluations, or even reframing the problem following new instructions or observed evidence. We instantiate this idea in Sara, a surrogate-augmented autoresearch agent, and lenz, a modular BoTorch-based backend that the agent can inspect and modify through a structured interface. Across synthetic and real-world benchmarks, Sara preserves the reliability of state-of-the-art BO without prior knowledge, outperforms LLM-based baselines, and uses natural-language priors to improve beyond standard BO. We further demonstrate the practical value of agentic BO in dynamic settings, where Sara reconfigures the full optimization problem on the fly as requirements change, a capability not previously available in standard BO.
