The integration of Large Language Models (LLMs) with evolutionary computation (EC) has introduced a promising paradigm for automating the design of metaheuristic algorithms. However, existing frameworks, such as the Large Language Model Evolutionary Algorithm (LLaMEA), often lack precise control over mutation mechanisms, leading to inefficiencies in solution space exploration and potentially suboptimal convergence. This paper introduces a novel approach to mutation control within LLM-driven evolutionary frameworks, inspired by theory of genetic algorithms. Specifically, we propose dynamic mutation prompts that adaptively regulate mutation rates, leveraging a heavy-tailed power-law distribution to balance exploration and exploitation. Experiments using GPT-3.5-turbo and GPT-4o models demonstrate that GPT-3.5-turbo fails to adhere to the specific mutation instructions, while GPT-4o is able to adapt its mutation based on the prompt engineered dynamic prompts. Further experiments show that the introduction of these dynamic rates can improve the convergence speed and adaptability of LLaMEA, when using GPT-4o. This work sets the starting point for better controlled LLM-based mutations in code optimization tasks, paving the way for further advancements in automated metaheuristic design.
It is well known that evolutionary algorithms (EAs) achieve peak performance only when their parameters are suitably tuned to the given problem. Even more, it is known that the best parameter values can change during the optimization process. Parameter control mechanisms are techniques developed to identify and to track these values. Recently, a series of rigorous theoretical works confirmed the superiority of several parameter control techniques over EAs with best possible static parameters. Among these results are examples for controlling the mutation rate of the $(1+λ)$~EA when optimizing t…
A key property underlying the success of evolutionary algorithms (EAs) is their global search behavior, which allows the algorithms to `jump' from a current state to other parts of the search space, thereby avoiding to get stuck in local optima. This property is obtained through a random choice of the radius at which offspring are sampled from previously evaluated solutions. It is well known that, thanks to this global search behavior, the probability that an EA using standard bit mutation finds a global optimum of an arbitrary function $f:\{0,1\}^n \to \mathbb{R}$ tends to one as the number o…
Evolutionary algorithms have been used in many ways to generate digital art. We study how evolutionary processes are used for evolutionary art and present a new approach to the transition of images. Our main idea is to define evolutionary processes for digital image transition, combining different variants of mutation and evolutionary mechanisms. We introduce box and strip mutation operators which are specifically designed for image transition. Our experimental results show that the process of an evolutionary algorithm in combination with these mutation operators can be used as a valuable way …
Differential Evolution (DE) proved to be one of the most successful evolutionary algorithms for global optimization purposes in continuous problems. The core operator in DE is mutation which can provide the algorithm with both exploration and exploitation. In this article, a new notation for DE is proposed which has a formula that can be utilized for generating and extracting novel mutations and by applying this new notation, four novel mutations are proposed. More importantly, by combining these novel trial vector generation strategies and four other well-known ones, we proposed Generalized M…
Differential Evolution (DE) is one of the most successful and powerful evolutionary algorithms for global optimization problem. The most important operator in this algorithm is mutation operator which parents are selected randomly to participate in it. Recently, numerous papers are tried to make this operator more intelligent by selection of parents for mutation intelligently. The intelligent selection for mutation vectors is performed by applying design space (also known as decision space) criterion or fitness space criterion, however, in both cases, half of valuable information of the proble…
A decomposition-based multi-objective evolutionary algorithm with a differential evolution variation operator (MOEA/D-DE) shows high performance on challenging multi-objective problems (MOPs). The DE mutation consists of three key components: a mutation strategy, an index selection method for parent individuals, and a bound-handling method. However, the configuration of the DE mutation operator that should be used for MOEA/D-DE has not been thoroughly investigated in the literature. This configuration choice confuses researchers and users of MOEA/D-DE. To address this issue, we present a revie…
We propose a new way to self-adjust the mutation rate in population-based evolutionary algorithms in discrete search spaces. Roughly speaking, it consists of creating half the offspring with a mutation rate that is twice the current mutation rate and the other half with half the current rate. The mutation rate is then updated to the rate used in that subpopulation which contains the best offspring. We analyze how the $(1+λ)$ evolutionary algorithm with this self-adjusting mutation rate optimizes the OneMax test function. We prove that this dynamic version of the $(1+λ)$ EA finds the optimum in…
Differential evolution (DE) is an effective population-based metaheuristic algorithm for solving complex optimisation problems. However, the performance of DE is sensitive to the mutation operator. In this paper, we propose a novel DE algorithm, Clu-DE, that improves the efficacy of DE using a novel clustering-based mutation operator. First, we find, using a clustering algorithm, a winner cluster in search space and select the best candidate solution in this cluster as the base vector in the mutation operator. Then, an updating scheme is introduced to include new candidate solutions in the cur…
The performance of a Multiobjective Evolutionary Algorithm (MOEA) is crucially dependent on the parameter setting of the operators. The most desired control of such parameters presents the characteristic of adaptiveness, i.e., the capacity of changing the value of the parameter, in distinct stages of the evolutionary process, using feedbacks from the search for determining the direction and/or magnitude of changing. Given the great popularity of the algorithm NSGA-II, the objective of this research is to create adaptive controls for each parameter existing in this MOEA. With these controls, we…
Currently, the genetic programming version of the gene-pool optimal mixing evolutionary algorithm (GP-GOMEA) is among the top-performing algorithms for symbolic regression (SR). A key strength of GP-GOMEA is its way of performing variation, which dynamically adapts to the emergence of patterns in the population. However, GP-GOMEA lacks a mechanism to optimize coefficients. In this paper, we study how fairly simple approaches for optimizing coefficients can be integrated into GP-GOMEA. In particular, we considered two variants of Gaussian coefficient mutation. We performed experiments using dif…
This paper examines three generic strategies for improving the performance of neuro-evolution techniques aimed at evolving convolutional neural networks (CNNs). These were implemented as part of the Evolutionary eXploration of Augmenting Convolutional Topologies (EXACT) algorithm. EXACT evolves arbitrary convolutional neural networks (CNNs) with goals of better discovering and understanding new effective architectures of CNNs for machine learning tasks and to potentially automate the process of network design and selection. The strategies examined are node-level mutation operations, epigenetic…
Explaining to what extent the real power of genetic algorithms lies in the ability of crossover to recombine individuals into higher quality solutions is an important problem in evolutionary computation. In this paper we show how the interplay between mutation and crossover can make genetic algorithms hillclimb faster than their mutation-only counterparts. We devise a Markov Chain framework that allows to rigorously prove an upper bound on the runtime of standard steady state genetic algorithms to hillclimb the OneMax function. The bound establishes that the steady-state genetic algorithms are…
Evolutionary algorithms (EAs) are general-purpose problem solvers that usually perform an unbiased search. This is reasonable and desirable in a black-box scenario. For combinatorial optimization problems, often more knowledge about the structure of optimal solutions is given, which can be leveraged by means of biased search operators. We consider the Minimum Spanning Tree (MST) problem in a single- and multi-objective version, and introduce a biased mutation, which puts more emphasis on the selection of edges of low rank in terms of low domination number. We present example graphs where the b…
Among many evolutionary algorithms, differential evolution (DE) has received much attention over the last two decades. DE is a simple yet powerful evolutionary algorithm that has been used successfully to optimize various real-world problems. Since it was introduced, many researchers have developed new methods for DE, and one of them makes use of a mutation based on the Cauchy distribution to increase the convergence speed of DE. The method monitors the results of each individual in the selection operator and performs the Cauchy mutation on consecutively failed individuals, which generates mut…
Most evolutionary algorithms have parameters, which allow a great flexibility in controlling their behavior and adapting them to new problems. To achieve the best performance, it is often needed to control some of the parameters during optimization, which gave rise to various parameter control methods. In recent works, however, similar advantages have been shown, and even proven, for sampling parameter values from certain, often heavy-tailed, fixed distributions. This produced a family of algorithms currently known as "fast evolution strategies" and "fast genetic algorithms". However, only lit…
Differential evolution (DE) has competitive performance on constrained optimization problems (COPs), which targets at searching for global optimal solution without violating the constraints. Generally, researchers pay more attention on avoiding violating the constraints than better objective function value. To achieve the aim of searching the feasible solutions accurately, an adaptive population size method and an adaptive mutation strategy are proposed in the paper. The adaptive population method is similar to a state switch which controls the exploring state and exploiting state according to…
Generating diverse populations of high quality solutions has gained interest as a promising extension to the traditional optimization tasks. This work contributes to this line of research with an investigation on evolutionary diversity optimization for three of the most well-studied permutation problems, namely the Traveling Salesperson Problem (TSP), both symmetric and asymmetric variants, and Quadratic Assignment Problem (QAP). It includes an analysis of the worst-case performance of a simple mutation-only evolutionary algorithm with different mutation operators, using an established diversi…
The interplay between mutation and selection plays a fundamental role in the behaviour of evolutionary algorithms (EAs). However, this interplay is still not completely understood. This paper presents a rigorous runtime analysis of a non-elitist population-based EA that uses the linear ranking selection mechanism. The analysis focuses on how the balance between parameter $η$, controlling the selection pressure in linear ranking, and parameter $χ$ controlling the bit-wise mutation rate, impacts the runtime of the algorithm. The results point out situations where a correct balance between select…
Evolutionary Algorithms (EAs) and other randomized search heuristics are often considered as unbiased algorithms that are invariant with respect to different transformations of the underlying search space. However, if a certain amount of domain knowledge is available the use of biased search operators in EAs becomes viable. We consider a simple (1+1) EA for binary search spaces and analyze an asymmetric mutation operator that can treat zero- and one-bits differently. This operator extends previous work by Jansen and Sudholt (ECJ 18(1), 2010) by allowing the operator asymmetry to vary according…
Differential evolution (DE) algorithm with a small population size is called Micro-DE (MDE). A small population size decreases the computational complexity but also reduces the exploration ability of DE by limiting the population diversity. In this paper, we propose the idea of combining ensemble mutation scheme selection and opposition-based learning concepts to enhance the diversity of population in MDE at mutation and selection stages. The proposed algorithm enhances the diversity of population by generating a random mutation scale factor per individual and per dimension, randomly assigning…
We analyze the performance of the 2-rate $(1+λ)$ Evolutionary Algorithm (EA) with self-adjusting mutation rate control, its 3-rate counterpart, and a $(1+λ)$~EA variant using multiplicative update rules on the OneMax problem. We compare their efficiency for offspring population sizes ranging up to $λ=3,200$ and problem sizes up to $n=100,000$. Our empirical results show that the ranking of the algorithms is very consistent across all tested dimensions, but strongly depends on the population size. While for small values of $λ$ the 2-rate EA performs best, the multiplicative updates become super…
Despite many successful applications, Cartesian Genetic Programming (CGP) suffers from limited scalability, especially when used for evolutionary circuit design. Considering the multiplier design problem, for example, the 5x5-bit multiplier represents the most complex circuit evolved from a randomly generated initial population. The efficiency of CGP highly depends on the performance of the point mutation operator, however, this operator is purely stochastic. This contrasts with the recent developments in Genetic Programming (GP), where advanced informed approaches such as semantic-aware opera…
We consider the expected runtime of non-elitist evolutionary algorithms (EAs), when they are applied to a family of fitness functions with a plateau of second-best fitness in a Hamming ball of radius r around a unique global optimum. On one hand, using the level-based theorems, we obtain polynomial upper bounds on the expected runtime for some modes of non-elitist EA based on unbiased mutation and the bitwise mutation in particular. On the other hand, we show that the EA with fitness proportionate selection is inefficient if the bitwise mutation is used with the standard settings of mutation p…
The differential evolution (DE) algorithm suffers from high computational time due to slow nature of evaluation. In contrast, micro-DE (MDE) algorithms employ a very small population size, which can converge faster to a reasonable solution. However, these algorithms are vulnerable to a premature convergence as well as to high risk of stagnation. In this paper, MDE algorithm with vectorized random mutation factor (MDEVM) is proposed, which utilizes the small size population benefit while empowers the exploration ability of mutation factor through randomizing it in the decision variable level. T…
Self-adaptive parameters are increasingly used in the field of Evolutionary Robotics, as they allow key evolutionary rates to vary autonomously in a context-sensitive manner throughout the optimisation process. A significant limitation to self-adaptive mutation is that rates can be set unfavourably, which hinders convergence. Rate restarts are typically employed to remedy this, but thus far have only been applied in Evolutionary Robotics for mutation-only algorithms. This paper focuses on the level at which evolutionary rate restarts are applied in population-based algorithms with more than 1 …
Neuroevolution is a promising area of research that combines evolutionary algorithms with neural networks. A popular subclass of neuroevolutionary methods, called evolution strategies, relies on dense noise perturbations to mutate networks, which can be sample inefficient and challenging for large models with millions of parameters. We introduce an approach to alleviating this problem by decomposing dense mutations into low-dimensional subspaces. Restricting mutations in this way can significantly reduce variance as networks can handle stronger perturbations while maintaining performance, whic…
ISAE-SUPAERO, Université de Toulouse（高等航空航天学院，图卢兹大学）；ENAC, Université de Toulouse（国立民用航空学院，图卢兹大学）
We study how Reinforcement Learning can be employed to optimally control parameters in evolutionary algorithms. We control the mutation probability of a (1+1) evolutionary algorithm on the OneMax function. This problem is modeled as a Markov Decision Process and solved with Value Iteration via the known transition probabilities. It is then solved via Q-Learning, a Reinforcement Learning algorithm, where the exact transition probabilities are not needed. This approach also allows previous expert or empirical knowledge to be included into learning. It opens new perspectives, both formally and co…
Two meta-evolutionary optimization strategies described in this paper accelerate the convergence of evolutionary programming algorithms while still retaining much of their ability to deal with multi-modal problems. The strategies, called directional mutation and recorded step in this paper, can operate independently but together they greatly enhance the ability of evolutionary programming algorithms to deal with fitness landscapes characterized by long narrow valleys. The directional mutation aspect of this combined method uses correlated meta-mutation but does not introduce a full covariance …
Harvard Medical School-MIT（哈佛医学院-麻省理工学院）；Moravian Academy Upper School（摩拉维亚学院高中部）
In recent years, optimization problems have become increasingly more prevalent due to the need for more powerful computational methods. With the more recent advent of technology such as artificial intelligence, new metaheuristics are needed that enhance the capabilities of classical algorithms. More recently, researchers have been looking at Charles Darwin's theory of natural selection and evolution as a means of enhancing current approaches using machine learning. In 1960, the first genetic algorithm was developed by John H. Holland and his student. We explore the mathematical intuition of th…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cognizant AI Labs（高知特人工智能实验室）；Sony AI（索尼人工智能）
Evolutionary algorithms are sensitive to the mutation rate (MR); no single value of this parameter works well across domains. Self-adaptive MR approaches have been proposed but they tend to be brittle: Sometimes they decay the MR to zero, thus halting evolution. To make self-adaptive MR robust, this paper introduces the Group Elite Selection of Mutation Rates (GESMR) algorithm. GESMR co-evolves a population of solutions and a population of MRs, such that each MR is assigned to a group of solutions. The resulting best mutational change in the group, instead of average mutational change, is used…
Natural selection explains how life has evolved over millions of years from more primitive forms. The speed at which this happens, however, has sometimes defied formal explanations when based on random (uniformly distributed) mutations. Here we investigate the application of a simplicity bias based on a natural but algorithmic distribution of mutations (no recombination) in various examples, particularly binary matrices in order to compare evolutionary convergence rates. Results both on synthetic and on small biological examples indicate an accelerated rate when mutations are not statistical u…
University of Malta（马耳他大学）；New York University（纽约大学）
Search-based procedural content generation (PCG) is a well-known method for level generation in games. Its key advantage is that it is generic and able to satisfy functional constraints. However, due to the heavy computational costs to run these algorithms online, search-based PCG is rarely utilized for real-time generation. In this paper, we introduce mutation models, a new type of iterative level generator based on machine learning. We train a model to imitate the evolutionary process and use the trained model to generate levels. This trained model is able to modify noisy levels sequentially…
