arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.01192cs.LG
收藏

Deep-ELA:基于自监督预训练Transformer的单目标和多目标连续优化问题深度探索性景观分析

Deep-ELA: Deep Exploratory Landscape Analysis with Self-Supervised Pretrained Transformers for Single- and Multi-Objective Continuous Optimization Problems

Paderborn University(帕德博恩大学);TU Dresden(德累斯顿工业大学);ScaDS.AI Dresden/Leipzig(德累斯顿/莱比锡数据科学与人工智能中心);University of Twente(特文特大学)

Moritz Vinzent Seiler, Pascal Kerschke, Heike Trautmann

AI总结:

针对ELA特征存在强相关性且难以用于多目标优化的问题,提出混合方法Deep-ELA,通过在数百万随机生成的优化问题上预训练四个Transformer学习景观深度表征,开箱即用或微调用于单/多目标连续优化问题的分析与算法行为理解。

摘要:

在许多近期研究中,探索性景观分析(ELA)特征对单目标连续优化问题进行数值表征的潜力已得到证实。这些数值特征为连续优化问题上的各类机器学习任务提供了输入,包括但不限于高层属性预测、自动化算法选择和自动化算法配置。据我们所知,若没有ELA特征,对单目标连续优化问题特征的分析与理解将非常有限。然而,尽管过往多项研究证明了其有用性,ELA特征仍存在若干缺陷。特别是:(1.) 多个特征之间存在强相关性;(2.) 其在多目标连续优化问题上的适用性非常有限。作为补救措施,近期研究提出了基于深度学习的方法作为ELA的替代方案。例如,在这些研究中使用了点云Transformer来表征优化问题的适应度景观。然而,这些方法需要大量带标签的训练数据。在本研究中,我们提出了一种混合方法Deep-ELA,它结合了深度学习与ELA特征的优势。具体而言,我们在数百万个随机生成的优化问题上预训练了四个Transformer,以学习连续单目标和多目标优化问题景观的深度表征。我们提出的框架既可以开箱即用地分析单目标和多目标连续优化问题,也可以随后针对侧重于算法行为和问题理解的各种任务进行微调。

英文摘要:

In many recent works, the potential of Exploratory Landscape Analysis (ELA) features to numerically characterize, in particular, single-objective continuous optimization problems has been demonstrated. These numerical features provide the input for all kinds of machine learning tasks on continuous optimization problems, ranging, i.a., from High-level Property Prediction to Automated Algorithm Selection and Automated Algorithm Configuration. Without ELA features, analyzing and understanding the characteristics of single-objective continuous optimization problems is -- to the best of our knowledge -- very limited. Yet, despite their usefulness, as demonstrated in several past works, ELA features suffer from several drawbacks. These include, in particular, (1.) a strong correlation between multiple features, as well as (2.) its very limited applicability to multi-objective continuous optimization problems. As a remedy, recent works proposed deep learning-based approaches as alternatives to ELA. In these works, e.g., point-cloud transformers were used to characterize an optimization problem's fitness landscape. However, these approaches require a large amount of labeled training data. Within this work, we propose a hybrid approach, Deep-ELA, which combines (the benefits of) deep learning and ELA features. Specifically, we pre-trained four transformers on millions of randomly generated optimization problems to learn deep representations of the landscapes of continuous single- and multi-objective optimization problems. Our proposed framework can either be used out-of-the-box for analyzing single- and multi-objective continuous optimization problems, or subsequently fine-tuned to various tasks focussing on algorithm behavior and problem understanding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑