Deep-ELA: Deep Exploratory Landscape Analysis with Self-Supervised Pretrained Transformers for Single- and Multi-Objective Continuous Optimization Problems
Paderborn University(帕德博恩大学);TU Dresden(德累斯顿工业大学);ScaDS.AI Dresden/Leipzig(德累斯顿/莱比锡数据科学与人工智能中心);University of Twente(特文特大学)
Moritz Vinzent Seiler, Pascal Kerschke, Heike Trautmann
In many recent works, the potential of Exploratory Landscape Analysis (ELA) features to numerically characterize, in particular, single-objective continuous optimization problems has been demonstrated. These numerical features provide the input for all kinds of machine learning tasks on continuous optimization problems, ranging, i.a., from High-level Property Prediction to Automated Algorithm Selection and Automated Algorithm Configuration. Without ELA features, analyzing and understanding the characteristics of single-objective continuous optimization problems is -- to the best of our knowledge -- very limited.
Yet, despite their usefulness, as demonstrated in several past works, ELA features suffer from several drawbacks. These include, in particular, (1.) a strong correlation between multiple features, as well as (2.) its very limited applicability to multi-objective continuous optimization problems. As a remedy, recent works proposed deep learning-based approaches as alternatives to ELA. In these works, e.g., point-cloud transformers were used to characterize an optimization problem's fitness landscape. However, these approaches require a large amount of labeled training data.
Within this work, we propose a hybrid approach, Deep-ELA, which combines (the benefits of) deep learning and ELA features. Specifically, we pre-trained four transformers on millions of randomly generated optimization problems to learn deep representations of the landscapes of continuous single- and multi-objective optimization problems. Our proposed framework can either be used out-of-the-box for analyzing single- and multi-objective continuous optimization problems, or subsequently fine-tuned to various tasks focussing on algorithm behavior and problem understanding.
Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy
Thales Research & Technology(泰雷兹研究与技术公司);CNRS(法国国家科学研究中心);École Polytechnique(巴黎综合理工学院);Institut Polytechnique de Paris(巴黎理工学院);Sorbonne Université(索邦大学)
Towards Explainable Exploratory Landscape Analysis: Extreme Feature Selection for Classifying BBOB Functions
Thales Research & Technology(泰雷兹研究与技术公司);École Polytechnique(巴黎综合理工学院);Institut Polytechnique de Paris(巴黎理工学院);CNRS(法国国家科学研究中心);Sorbonne Université(索邦大学)
Does Dimensionality Reduction via Random Projections Preserve Landscape Features?
Leiden Institute of Advanced Computer Science, Leiden University(莱顿高级计算机科学研究所,莱顿大学);RTWH Aachen University(亚琛工业大学);Institute for Clarity in Documentation(文档清晰研究所);Inria Paris-Rocquencourt(巴黎-罗quentourt研究所);Rajiv Gandhi University(拉贾·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒研究实验室)
A foundation model for energy and radiation systems built on heterogeneous scientific interfaces
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Indian Institute of Technology Delhi(印度理工学院德里分校);Yardi School of Artificial Intelligence (ScAI)(雅迪人工智能学院);National Center for Supercomputing Applications (NCSA)(国家超级计算应用中心)