arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.13892cs.LG
收藏

多目标分层优化与大语言模型

Multi-Objective Hierarchical Optimization with Large Language Models

University of Freiburg(弗赖堡大学);ELLIS Institute Tübingen(图宾根ELLIS研究所)

Andrej Schwanke, Lyubomir Ivanov, David Salinas, Frank Hutter, Arber Zela

AI总结:

本文提出利用大语言模型作为替代模型和候选采样器,结合结构化分层搜索策略,实现多目标优化,实验表明其在多个基准上表现优异。

摘要:

尽管在各种领域中被广泛应用,尤其是由于其强大的推理能力,大语言模型(LLMs)尚未成为驱动多目标优化的现成选择。传统策略在基准测试中表现优异,因其内在能力处理数值输入以及精心设计的建模选择,能够平衡探索和帕累托前沿利用,同时处理多个(冲突的)目标。在本文中,我们通过将LLMs用作替代模型和候选采样器,结合结构化的分层搜索策略,弥合这一差距。通过自适应地将输入空间划分为不相交的超矩形区域,并用复合评分函数对它们进行排序,我们将LLM的生成过程限制在特定的高潜力子空间中,从而使问题更容易解决,因为LLM无需推理问题的全局结构,只需局部处理。我们证明,在标准正则性假设下,我们的算法生成的候选解在豪斯多夫距离上收敛到真实的帕累托集。在经验上,它在标准的全局LLM多目标优化器上表现一致优于,并在合成和现实世界基准上与标准进化和贝叶斯优化算法相当。

英文摘要:

Despite their widespread adoption in various domains, especially due to their powerful reasoning capabilities, Large Language Models (LLMs) are not the off-the-shelf choice to drive multi-objective optimization yet. Conventional strategies rank high in benchmarks due to their intrinsic capabilities to handle numerical inputs and careful modelling choices that balance exploration and Pareto-front exploitation, as well as handle multiple (conflicting) objectives. In this paper, we close this gap by leveraging LLMs as surrogate models and candidate samplers inside a structured hierarchical search strategy. By adaptively partitioning the input space into disjoint hyperrectangular regions and ranking them with a composite score function, we restrict the generative process of the LLM to specific, high-potential sub-spaces, hence making the problem easier to solve as the LLM doesn't have to reason about the global structure of the problem, but only locally instead. We show that under standard regularity assumptions, our algorithm generates candidate solutions that converge to the true Pareto set in Hausdorff distance. Empirically, it consistently outperforms the global LLM-based multi-objective optimizer and is on par with standard evolutionary and Bayesian optimization algorithm on synthetic and real-world benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑