LLaMEA-SAGE: 通过可解释AI的结构反馈引导自动算法设计
LLaMEA-SAGE: Guiding Automated Algorithm Design with Structural Feedback from Explainable AI
- LIACS, Leiden University（莱顿大学信息科学研究中心）
- University of St Andrews（圣安德鲁大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
LLaMEA-SAGE通过可解释AI的结构反馈引导自动算法设计，提升搜索效率和性能。
AI中文摘要：
大型语言模型通过直接从自然语言提示生成优化算法，实现了自动算法设计（AAD）。虽然进化框架如LLaMEA在算法设计空间中展现出强大的探索能力，但其搜索动态完全由适应度反馈驱动，导致生成代码中大量信息未被利用。我们提出了一种机制，通过从生成算法的抽象语法树中提取图论和复杂性特征，构建反馈，基于在评估解决方案档案上学习的替代模型，用于引导AAD。利用可解释AI技术，我们识别出显著影响性能的特征，并将其转化为自然语言突变指令，以指导后续基于LLM的代码生成，而不限制表达性。我们提出了LLaMEA-SAGE，将其特征驱动的引导整合到LLaMEA中，并在多个基准上进行了评估。我们显示，在小规模受控实验中，所提出的结构化引导在相同性能方面比普通LLaMEA更快。在使用GECCO-MA-BBOB竞赛的MA-BBOB套件进行的更大规模实验中，我们的引导方法相比最先进的AAD方法表现出优越的性能。这些结果表明，从代码中获得的信号可以有效偏转LLM驱动的算法进化，弥合了代码结构与人类可理解的性能反馈之间的差距。
英文摘要：
Large language models have enabled automated algorithm design (AAD) by generating optimization algorithms directly from natural-language prompts. While evolutionary frameworks such as LLaMEA demonstrate strong exploratory capabilities across the algorithm design space, their search dynamics are entirely driven by fitness feedback, leaving substantial information about the generated code unused. We propose a mechanism for guiding AAD using feedback constructed from graph-theoretic and complexity features extracted from the abstract syntax trees of the generated algorithms, based on a surrogate model learned over an archive of evaluated solutions. Using explainable AI techniques, we identify features that substantially affect performance and translate them into natural-language mutation instructions that steer subsequent LLM-based code generation without restricting expressivity. We propose LLaMEA-SAGE, which integrates this feature-driven guidance into LLaMEA, and evaluate it across several benchmarks. We show that the proposed structured guidance achieves the same performance faster than vanilla LLaMEA in a small controlled experiment. In a larger-scale experiment using the MA-BBOB suite from the GECCO-MA-BBOB competition, our guided approach achieves superior performance compared to state-of-the-art AAD methods. These results demonstrate that signals derived from code can effectively bias LLM-driven algorithm evolution, bridging the gap between code structure and human-understandable performance feedback in automated algorithm design.
