Evolutionary algorithms (EAs), simulating the evolution process of natural species, are used to solve optimization problems. Crossover (also called recombination), originated from simulating the chromosome exchange phenomena in zoogamy reproduction, is widely employed in EAs to generate offspring solutions, of which the effectiveness has been examined empirically in applications. However, due to the irregularity of crossover operators and the complicated interactions to mutation, crossover operators are hard to analyze and thus have few theoretical results. Therefore, analyzing crossover not only helps in understanding EAs, but also helps in developing novel techniques for analyzing sophisticated metaheuristic algorithms.
In this paper, we derive the General Markov Chain Switching Theorem (GMCST) to facilitate theoretical studies of crossover-enabled EAs. The theorem allows us to analyze the running time of a sophisticated EA from an easy-to-analyze EA. Using this tool, we analyze EAs with several crossover operators on the LeadingOnes and OneMax problems, which are noticeably two well studied problems for mutation-only EAs but with few results for crossover-enabled EAs. We first derive the bounds of running time of the (2+2)-EA with crossover operators; then we study the running time gap between the mutation-only (2:2)-EA and the (2:2)-EA with crossover operators; finally, we develop strategies that apply crossover operators only when necessary, which improve from the mutation-only as well as the crossover-all-the-time (2:2)-EA. The theoretical results are verified by experiments.
Although differential evolution (DE) algorithms perform well on a large variety of complicated optimization problems, only a few theoretical studies are focused on the working principle of DE algorithms. To make the first attempt to reveal the function of binomial crossover, this paper aims to answer whether it can reduce the approximation error of evolutionary algorithms. By investigating the expected approximation error and the probability of not finding the optimum, we conduct a case study comparing two evolutionary algorithms with and without binomial crossover on two classical benchmark p…
This paper proposes a generalized Hybrid Real-coded Quantum Evolutionary Algorithm (HRCQEA) for optimizing complex functions as well as combinatorial optimization. The main idea of HRCQEA is to devise a new technique for mutation and crossover operators. Using the evolutionary equation of PSO a Single-Multiple gene Mutation (SMM) is designed and the concept of Arithmetic Crossover (AC) is used in the new Crossover operator. In HRCQEA, each triploid chromosome represents a particle and the position of the particle is updated using SMM and Quantum Rotation Gate (QRG), which can make the balance …
In this paper we propose a crossover operator for evolutionary algorithms with real values that is based on the statistical theory of population distributions. The operator is based on the theoretical distribution of the values of the genes of the best individuals in the population. The proposed operator takes into account the localization and dispersion features of the best individuals of the population with the objective that these features would be inherited by the offspring. Our aim is the optimization of the balance between exploration and exploitation in the search process. In order to t…
Crossover and mutation are the two main operators that lead to new solutions in evolutionary approaches. In this article, a new method of performing the crossover phase is presented. The problem of choice is evolutionary decision tree construction. The method aims at finding such individuals that together complement each other. Hence we say that they are diversely specialized. We propose the way of calculating the so-called complementary fitness. In several empirical experiments, we evaluate the efficacy of the method proposed in four variants and compare it to a fitness-rank-based approach. O…
Generative Adversarial Networks (GAN) is an adversarial model, and it has been demonstrated to be effective for various generative tasks. However, GAN and its variants also suffer from many training problems, such as mode collapse and gradient vanish. In this paper, we firstly propose a general crossover operator, which can be widely applied to GANs using evolutionary strategies. Then we design an evolutionary GAN framework C-GAN based on it. And we combine the crossover operator with evolutionary generative adversarial networks (EGAN) to implement the evolutionary generative adversarial netwo…
Jilin University（吉林大学）；Northeast Normal University（东北师范大学）
The training of generative adversarial networks (GANs) is usually vulnerable to mode collapse and vanishing gradients. The evolutionary generative adversarial network (E-GAN) attempts to alleviate these issues by optimizing the learning strategy with multiple loss functions. It uses a learning-based evolutionary framework, which develops new mutation operators specifically for general deep neural networks. However, the evaluation mechanism in the fitness function of E-GAN cannot truly reflect the adaptability of individuals to their environment, leading to an inaccurate assessment of the diver…
The notion of building blocks can be related to the structure of the offspring probability distribution: loci of which variability is strongly correlated constitute a building block. We call this correlated exploration. With this background we analyze the structure of the offspring probability distribution, or exploration distribution, for a GA with mutation only, a crossover GA, and an Estimation-Of-Distribution Algorithm (EDA). The results allow a precise characterization of the structure of the crossover exploration distribution. Essentially, the crossover operator destroys mutual informati…
The all-pairs shortest path problem is the first non-artificial problem for which it was shown that adding crossover can significantly speed up a mutation-only evolutionary algorithm. Recently, the analysis of this algorithm was refined and it was shown to have an expected optimization time (w.r.t. the number of fitness evaluations) of $Θ(n^{3.25}(\log n)^{0.25})$. In contrast to this simple algorithm, evolutionary algorithms used in practice usually employ refined recombination strategies in order to avoid the creation of infeasible offspring. We study extensions of the basic algorithm by two…
Evolutionary algorithms have been widely used for a range of stochastic optimization problems in order to address complex real-world optimization problems. We consider the knapsack problem where the profits involve uncertainties. Such a stochastic setting reflects important real-world scenarios where the profit that can be realized is uncertain. We introduce different ways of dealing with stochastic profits based on tail inequalities such as Chebyshev's inequality and Hoeffding bounds that allow to limit the impact of uncertainties. We examine simple evolutionary algorithms and the use of heav…
Explaining to what extent the real power of genetic algorithms lies in the ability of crossover to recombine individuals into higher quality solutions is an important problem in evolutionary computation. In this paper we show how the interplay between mutation and crossover can make genetic algorithms hillclimb faster than their mutation-only counterparts. We devise a Markov Chain framework that allows to rigorously prove an upper bound on the runtime of standard steady state genetic algorithms to hillclimb the OneMax function. The bound establishes that the steady-state genetic algorithms are…
This manuscript contains an outline of lectures course "Evolutionary Algorithms" read by the author. The course covers Canonic Genetic Algorithm and various other genetic algorithms as well as evolutionary strategies, genetic programming, tabu search and the class of evolutionary algorithms in general. Some facts, such as the Rotation Property of crossover, the Schemata Theorem, GA performance as a local search and "almost surely" convergence of evolutionary algorithms are given with complete proofs. The text is in Russian.
In the paper, an evolutionary approach to test generation for functional BIST is considered. The aim of the proposed scheme is to minimize the test data volume by allowing the device's microprogram to test its logic, providing an observation structure to the system, and generating appropriate test data for the given architecture. Two methods of deriving a deterministic test set at functional level are suggested. The first method is based on the classical genetic algorithm with binary and arithmetic crossover and mutation operators. The second one uses genetic programming, where test is represe…
The chance-constrained knapsack problem is a variant of the classical knapsack problem where each item has a weight distribution instead of a deterministic weight. The objective is to maximize the total profit of the selected items under the condition that the weight of the selected items only exceeds the given weight bound with a small probability of $α$. In this paper, consider problem-specific single-objective and multi-objective approaches for the problem. We examine the use of heavy-tail mutations and introduce a problem-specific crossover operator to deal with the chance-constrained knap…
There has been a variety of crossover operators proposed for Real-Coded Genetic Algorithms (RCGAs), which recombine values from the same location in pairs of strings. In this article we present a recombination operator for RC- GAs that selects the locations randomly in both parents, and compare it to mainstream crossover operators in a set of experiments on a range of standard multidimensional optimization problems and a clustering problem. We present two variants of the operator, either selecting both bits uniformly at random in the strings, or sampling the second bit from a normal distributi…
We re-investigate a fundamental question: how effective is crossover in Genetic Algorithms in combining building blocks of good solutions? Although this has been discussed controversially for decades, we are still lacking a rigorous and intuitive answer. We provide such answers for royal road functions and OneMax, where every bit is a building block. For the latter we show that using crossover makes every ($μ$+$λ$) Genetic Algorithm at least twice as fast as the fastest evolutionary algorithm using only standard bit mutation, up to small-order terms and for moderate $μ$ and $λ$. Crossover is b…
Erbil Polytechnic University（埃尔比勒理工大学）；Charmo University（查尔莫大学）；University of Kurdistan Hewler（库尔德斯坦休勒大学）
The performance of most evolutionary metaheuristic algorithms relays on various operatives. One of them is the crossover operator, which is divided into two types: application dependent and application independent crossover operators. These standards always help to choose the best-fitted point in the evolutionary algorithm process. High efficiency of crossover operators allows engineers to minimize errors in engineering application optimization while saving time and avoiding costly. There are two crucial objectives behind this paper; at first, it is an overview of crossover standards classific…
It is known that the evolutionary algorithm $(1+1)$-EA with mutation rate $c/n$ optimises every monotone function efficiently if $c<1$, and needs exponential time on some monotone functions (HotTopic functions) if $c\geq 2.2$. We study the same question for a large variety of algorithms, particularly for $(1+λ)$-EA, $(μ+1)$-EA, $(μ+1)$-GA, their fast counterparts like fast $(1+1)$-EA, and for $(1+(λ,λ))$-GA. We find that all considered mutation-based algorithms show a similar dichotomy for HotTopic functions, or even for all monotone functions. For the $(1+(λ,λ))$-GA, this dichotomy is in the …
The use of balanced crossover operators in Genetic Algorithms (GA) ensures that the binary strings generated as offsprings have the same Hamming weight of the parents, a constraint which is sought in certain discrete optimization problems. Although this method reduces the size of the search space, the resulting fitness landscape often becomes more difficult for the GA to explore and to discover optimal solutions. This issue has been studied in this paper by applying an adaptive bias strategy to a counter-based crossover operator that introduces unbalancedness in the offspring with a certain pr…
Genetic Algorithm is an evolutionary algorithm and a metaheuristic that was introduced to overcome the failure of gradient based method in solving the optimization and search problems. The purpose of this paper is to evaluate the impact on the convergence of Genetic Algorithm vis-a-vis 0/1 knapsack. By keeping the number of generations and the initial population fixed, different crossover methods like one point crossover and two-point crossover were evaluated and juxtaposed with each other. In addition to this, the impact of different selection methods like rank-selection, roulette wheel and t…
Evolutionary algorithms (EA) have been widely accepted as efficient solvers for complex real world optimization problems, including engineering optimization. However, real world optimization problems often involve uncertain environment including noisy and/or dynamic environments, which pose major challenges to EA-based optimization. The presence of noise interferes with the evaluation and the selection process of EA, and thus adversely affects its performance. In addition, as presence of noise poses challenges to the evaluation of the fitness function, it may need to be estimated instead of be…
A novel multi-agent evolutionary robotics (ER) based framework, inspired by competitive evolutionary environments in nature, is demonstrated for training Spiking Neural Networks (SNN). The weights of a population of SNNs along with morphological parameters of bots they control in the ER environment are treated as phenotypes. Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment. While the bots and their SNNs are given no explicit reward to survive or reproduce via any loss fun…
In this paper, we present that genotype-phenotype mapping can be theoretically interpreted using the concept of quotient space in mathematics. Quotient space can be considered as mathematically-defined phenotype space in the evolutionary computation theory. The quotient geometric crossover has the effect of reducing the search space actually searched by geometric crossover, and it introduces problem knowledge in the search by using a distance better tailored to the specific solution interpretation. Quotient geometric crossovers are directly applied to the genotype space but they have the effec…
In evolutionary algorithms, genetic operators iteratively generate new offspring which constitute a potentially valuable set of search history. To boost the performance of crossover in real-coded genetic algorithm (RCGA), in this paper we propose to exploit the search history cached so far in an online style during the iteration. Specifically, survivor individuals over past few generations are collected and stored in the archive to form the search history. We introduce a simple yet effective crossover model driven by the search history (abbreviated as SHX). In particular, the search history is…
In the context of evolutionary quantum computing in the literal meaning, a quantum crossover operation has not been introduced so far. Here, we introduce a novel quantum genetic algorithm which has a quantum crossover procedure performing crossovers among all chromosomes in parallel for each generation. A complexity analysis shows that a quadratic speedup is achieved over its classical counterpart in the dominant factor of the run time to handle each generation.
Real-valued genotypes together with the variation operators, mutation and crossover, constitute some of the fundamental building blocks of Evolutionary Algorithms. Real-valued genotypes are utilized in a broad range of contexts, from weights in Artificial Neural Networks to parameters in robot control systems. Shared between most uses of real-valued genomes is the need for limiting the range of individual parameters to allowable bounds. In this paper we will illustrate the challenge of limiting the parameters of real-valued genomes and analyse the most promising method to properly limit these …
Evolutionary algorithms usually explore a search space of solutions by means of crossover and mutation. While a mutation consists of a small, local modification of a solution, crossover mixes the genetic information of two solutions to compute a new one. For model-driven optimization (MDO), where models directly serve as possible solutions (instead of first transforming them into another representation), only recently a generic crossover operator has been developed. Using graphs as a formal foundation for models, we further refine this operator in such a way that additional well-formedness con…
For theoretical analyses there are two specifics distinguishing GP from many other areas of evolutionary computation. First, the variable size representations, in particular yielding a possible bloat (i.e. the growth of individuals with redundant parts). Second, the role and realization of crossover, which is particularly central in GP due to the tree-based representation. Whereas some theoretical work on GP has studied the effects of bloat, crossover had a surprisingly little share in this work. We analyze a simple crossover operator in combination with local search, where a preference for sm…
In practise, it is often desirable to provide the decision-maker with a rich set of diverse solutions of decent quality instead of just a single solution. In this paper we study evolutionary diversity optimization for the knapsack problem (KP). Our goal is to evolve a population of solutions that all have a profit of at least $(1-\varepsilon)\cdot OPT$, where OPT is the value of an optimal solution. Furthermore, they should differ in structure with respect to an entropy-based diversity measure. To this end we propose a simple $(μ+1)$-EA with initial approximate solutions calculated by a well-k…
Evolutionary Architecture Search through Grammar-Based Sequence Alignment
Carlos III University of Madrid（马德里卡洛斯三世大学）；University of Edinburgh（爱丁堡大学）；Instituto de Investigación Sanitaria Gregorio Marañón（Gregorio Marañón卫生研究院）；Helmholtz-Zentrum Berlin für Materialien und Energie（柏林材料与能源研究中心）；ELLIS Institute Tübingen（图宾根ELLIS研究所）；University of Glasgow（格拉斯哥大学）
Khulna University of Engineering and Technology（库尔纳工程技术大学）
This paper presents two different evolutionary systems - Evolutionary Programming Network (EPNet) and Novel Evolutions Strategy (NES) Algorithm. EPNet does both training and architecture evolution simultaneously, whereas NES does a fixed network and only trains the network. Five mutation operators proposed in EPNet to reflect the emphasis on evolving ANNs behaviors. Close behavioral links between parents and their offspring are maintained by various mutations, such as partial training and node splitting. On the other hand, NES uses two new genetic operators - subpopulation-based max-mean arith…
Node Preservation and its Effect on Crossover in Cartesian Genetic Programming
Department of Computer Science（计算机科学系）；BEACON Center for the Study of Evolution in Action Michigan State University, East Lansing, MI 48824, USA（进化机制行动研究中心 密歇根州立大学 埃文斯兰斯）
Efficient Evolutionary Search Over Chemical Space with Large Language Models
Georgia Institute of Technology（佐治亚理工学院）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Massachusetts Institute of Technology（麻省理工学院）；University of Wuppertal（伍珀塔尔大学）；Deep Principle Inc.（Deep Principle公司）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Cornell University（康奈尔大学）；Université de Montréal（蒙特利尔大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cognizant Technology Solutions（高知特技术解决方案公司）
The choice of activation function can have a large effect on the performance of a neural network. While there have been some attempts to hand-engineer novel activation functions, the Rectified Linear Unit (ReLU) remains the most commonly-used in practice. This paper shows that evolutionary algorithms can discover novel activation functions that outperform ReLU. A tree-based search space of candidate activation functions is defined and explored with mutation, crossover, and exhaustive search. Experiments on training wide residual networks on the CIFAR-10 and CIFAR-100 image datasets show that t…
Hanoi University of Science and Technology（河内理工大学）；Tay Bac University（西北大学）
Minimum Routing Cost Clustered Tree Problem (CluMRCT) is applied in various fields in both theory and application. Because the CluMRCT is NP-Hard, the approximate approaches are suitable to find the solution for this problem. Recently, Multifactorial Evolutionary Algorithm (MFEA) has emerged as one of the most efficient approximation algorithms to deal with many different kinds of problems. Therefore, this paper studies to apply MFEA for solving CluMRCT problems. In the proposed MFEA, we focus on crossover and mutation operators which create a valid solution of CluMRCT problem in two levels: f…
Coronavirus Optimization Algorithm: A Success-History Adaptive Evolutionary Framework with Archive-Assisted Search and Stagnation Recovery for Global Optimization
