基于大语言模型的在线自动化算法设计

中文摘要 AI 辅助

大语言模型（LLMs）通过推理和代码合成实现了自动化算法设计（AAD）。然而，现有大多数基于LLM的AAD方法将算法设计与目标优化分离，即使优化状态不断演变，也部署固定设计。传统自适应优化器能够响应此类变化，但其调整仍局限于预定义的参数、算子或策略。为解决这些局限性，我们引入了基于LLM的在线AAD，这是一种新颖的优化范式，将算法本身视为状态相关的决策变量。在每个阶段，LLM智能体根据当前优化状态合成具有新行为逻辑的算法。执行生成的算法推进搜索，并为后续设计提供反馈，从而将算法设计与目标优化耦合，无需单独的离线算法预训练阶段。为实现这一范式，我们提出了OnDesign，一个多智能体框架，它协调相互竞争的设计视角以合成可执行算法，并利用执行反馈来改进对运行时证据的解释，以用于后续设计。我们在三个场景中评估了OnDesign，涵盖两种主流黑箱优化范式：贝叶斯优化、进化连续优化和进化混合变量优化。在六个基准测试套件和一个跨多个问题维度的工程问题上的大量实验表明，其整体性能优于传统优化器和离线基于LLM的AAD方法。

英文摘要

Large language models (LLMs) enable automated algorithm design (AAD) through reasoning and code synthesis. However, most existing LLM-based AAD methods separate algorithm design from target optimization, deploying a fixed design even as the optimization state evolves. Conventional adaptive optimizers can respond to such changes, but their adjustments remain confined to predefined parameters, operators, or strategies. To address these limitations, we introduce online LLM-based AAD, a novel optimization paradigm that treats the algorithm itself as a state-dependent decision variable. At each stage, LLM agents synthesize an algorithm with new behavior logic from the current optimization state. Executing the generated algorithm advances the search and provides feedback for subsequent designs, coupling algorithm design with target optimization without requiring a separate offline algorithm pretraining stage. To implement this paradigm, we propose OnDesign, a multi-agent framework that reconciles competing design perspectives to synthesize executable algorithms and uses execution feedback to refine how runtime evidence is interpreted for subsequent designs. We evaluate OnDesign across two mainstream black-box optimization paradigms on three scenarios: Bayesian optimization, evolutionary continuous optimization, and evolutionary mixed-variable optimization. Extensive experiments on six benchmark suites and one engineering problem across multiple problem dimensions demonstrate superior overall performance over conventional optimizers and offline LLM-based AAD methods.

发表机构

• The Hong Kong Polytechnic University（香港理工大学）
• Chongqing University（重庆大学）

机构由 AI 辅助整理，请以论文原文为准。