贝叶斯优化结合大语言模型丰富的辅助信息
Bayesian Optimization with Rich Auxiliary Information via LLMs
- Carnegie Mellon University（卡内基梅隆大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出利用大型语言模型整合丰富辅助信息（如训练曲线、专家笔记）到贝叶斯优化中，开发三种方法，在超参数优化和核聚变任务上优于现有方法。
AI中文摘要：
贝叶斯优化（BO）广泛用于优化昂贵的黑盒函数，然而许多现实世界的优化问题包含的信息远比单纯的函数评估丰富得多。例如，超参数优化中的训练曲线、科学实验中的专家笔记和图像，以及关于最优值可能位于何处的先验知识。我们表明，大型语言模型（LLMs）能够有效利用此类丰富的辅助信息来指导优化。受这些发现的启发，我们开发了三种方法，利用LLMs将辅助信息整合到BO中。在超参数优化基准测试和一个真实的核聚变优化任务中，我们的方法始终优于标准BO和现有的基于LLM的优化方法。我们的结果证明了LLMs在BO中利用丰富辅助信息的有效性。
英文摘要：
Bayesian Optimization (BO) is widely used for optimizing expensive black-box functions, yet it typically reduces each expensive experiment to an input and its objective value, discarding much of the information the experiment produces. Such auxiliary information can include training curves in hyperparameter optimization, expert notes and images in scientific experimentation, or known facts about the specific optimization problem. Leveraging the ability of large language models (LLMs) to process diverse and unstructured information, we introduce two methods: AuxBO-Evolve, which uses auxiliary information to evolve beliefs over the location of the optimum, and AuxBO-Vicinity, which uses it to locally guide the acquisition function. Across synthetic tasks, hyperparameter optimization benchmarks, and a nuclear fusion task with unstructured scientist-written logs, our methods consistently improve optimization performance and outperform standard BO and existing LLM-based approaches. We further find that LLMs provide more effective probabilistic guidance when modeling likely maximizer locations rather than objective values pointwise. Overall, our results demonstrate that exploiting rich information beyond standard $(\mathbf{x}, y)$ interactions can substantially improve BO performance.
