A large number of engineering, science and computational problems have yet to be solved in a computationally efficient way. One of the emerging challenges is how evolving technologies grow towards autonomy and intelligent decision making. This leads to collection of large amounts of data from various sensing and measurement technologies, e.g., cameras, smart phones, health sensors, smart electricity meters, and environment sensors. Hence, it is imperative to develop efficient algorithms for generation, analysis, classification, and illustration of data. Meanwhile, data is structured purposefully through different representations, such as large-scale networks and graphs. We focus on data science as a crucial area, specifically focusing on a curse of dimensionality (CoD) which is due to the large amount of generated/sensed/collected data. This motivates researchers to think about optimization and to apply nature-inspired algorithms, such as evolutionary algorithms (EAs) to solve optimization problems. Although these algorithms look un-deterministic, they are robust enough to reach an optimal solution. Researchers do not adopt evolutionary algorithms unless they face a problem which is suffering from placement in local optimal solution, rather than global optimal solution. In this chapter, we first develop a clear and formal definition of the CoD problem, next we focus on feature extraction techniques and categories, then we provide a general overview of meta-heuristic algorithms, its terminology, and desirable properties of evolutionary algorithms.
University of Georgia（佐治亚大学）；Florida International University（佛罗里达国际大学）
In the past 30 years, scientists have searched nature, including animals and insects, and biology in order to discover, understand, and model solutions for solving large-scale science challenges. The study of bionics reveals that how the biological structures, functions found in nature have improved our modern technologies. In this study, we present our discovery of evolutionary and nature-inspired algorithms applications in Data Science and Data Analytics in three main topics of pre-processing, supervised algorithms, and unsupervised algorithms. Among all applications, in this study, we aim t…
Facebook Artificial Intelligence Research (FAIR)（脸书人工智能研究院（FAIR））；LAMSADE（LAMSADE实验室）；CNRS（法国国家科学研究中心）；Université Paris-Dauphine（巴黎多芬大学）；Université PSL（巴黎文理研究大学）
Choosing the right selection rate is a long standing issue in evolutionary computation. In the continuous unconstrained case, we prove mathematically that a single parent $μ=1$ leads to a sub-optimal simple regret in the case of the sphere function. We provide a theoretically-based selection rate $μ/λ$ that leads to better progress rates. With our choice of selection rate, we get a provable regret of order $O(λ^{-1})$ which has to be compared with $O(λ^{-2/d})$ in the case where $μ=1$. We complete our study with experiments to confirm our theoretical claims.
Santa Fe Institute（圣塔菲研究所）；University of New Mexico（新墨西哥大学）
In July of 2021, the Santa Fe Institute hosted a workshop on evolutionary computation as part of its Foundations of Intelligence in Natural and Artificial Systems project. This project seeks to advance the field of artificial intelligence by promoting interdisciplinary research on the nature of intelligence. The workshop brought together computer scientists and biologists to share their insights about the nature of evolution and the future of evolutionary computation. In this report, we summarize each of the talks and the subsequent discussions. We also draw out a number of key themes and iden…
The Wivace 2013 Electronic Proceedings in Theoretical Computer Science (EPTCS) contain some selected long and short articles accepted for the presentation at Wivace 2013 - Italian Workshop on Artificial Life and Evolutionary Computation, which was held at the University of Milan-Bicocca, Milan, on the 1st and 2nd of July, 2013.
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cognizant Technology Solutions（高知特科技解决方案公司）
The main power of artificial intelligence is not in modeling what we already know, but in creating solutions that are new. Such solutions exist in extremely large, high-dimensional, and complex search spaces. Population-based search techniques, i.e. variants of evolutionary computation, are well suited to finding them. These techniques make it possible to find creative solutions to practical problems in the real world, making creative AI through evolutionary computation the likely "next deep learning."
University of California Merced（加州大学默塞德分校）；CINVESTAV（国立理工学院高级研究中心）
This paper presents a framework for estimating the remaining useful life (RUL) of mechanical systems. The framework consists of a multi-layer perceptron and an evolutionary algorithm for optimizing the data-related parameters. The framework makes use of a strided time window to estimate the RUL for mechanical components. Tuning the data-related parameters can become a very time consuming task. The framework presented here automatically reshapes the data such that the efficiency of the model is increased. Furthermore, the complexity of the model is kept low, e.g. neural networks with few hidden…
The University of Texas at Austin（得克萨斯大学奥斯汀分校）；Cognizant Technology Solutions（高知特技术解决方案公司）
The main power of artificial intelligence is not in modeling what we already know, but in creating solutions that are new. Such solutions exist in extremely large, high-dimensional, and complex search spaces. Population-based search techniques, i.e. variants of evolutionary computation, are well suited to finding them. These techniques are also well positioned to take advantage of large-scale parallel computing resources, making creative AI through evolutionary computation the likely "next deep learning".
This effort examines the intersection of the emerging field of quantum computing and the more established field of evolutionary computation. The goal is to understand what benefits quantum computing might offer to computational intelligence and how computational intelligence paradigms might be implemented as quantum programs to be run on a future quantum computer. We critically examine proposed algorithms and methods for implementing computational intelligence paradigms, primarily focused on heuristic optimization methods including and related to evolutionary computation, with particular regar…
Recent developments in artificial intelligence and machine learning have spurred interest in the growing field of AI safety, which studies how to prevent human-harming accidents when deploying AI systems. This paper thus explores the intersection of AI safety with evolutionary computation, to show how safety issues arise in evolutionary computation and how understanding from evolutionary computational and biological evolution can inform the broader study of AI safety.
Reinforcement learning is increasingly finding success across domains where the problem can be represented as a Markov decision process. Evolutionary computation algorithms have also proven successful in this domain, exhibiting similar performance to the generally more complex reinforcement learning. Whilst there exist many open-source reinforcement learning and evolutionary computation libraries, no publicly available library combines the two approaches for enhanced comparison, cooperation, or visualization. To this end, we have created Pearl (https://github.com/LondonNode/Pearl), an open sou…
Evolutionary computation techniques have mostly been used to solve various optimization and learning problems successfully. Evolutionary algorithm is more effective to gain optimal solution(s) to solve complex problems than traditional methods. In case of problems with large set of parameters, evolutionary computation technique incurs a huge computational burden for a single processing unit. Taking this limitation into account, this paper presents a new distributed evolutionary computation technique, which decomposes decision vectors into smaller components and achieves optimal solution in a s…
This paper analises distributed evolutionary computation based on the Representational State Transfer (REST) protocol, which overlays a farming model on evolutionary computation. An approach to evolutionary distributed optimisation of multilayer perceptrons (MLP) using REST and language Perl has been done. In these experiments, a master-slave based evolutionary algorithm (EA) has been implemented, where slave processes evaluate the costly fitness function (training a MLP to solve a classification problem). Obtained results show that the parallel version of the developed programs obtains simila…
Resource-intensive computations are a major factor that limits the effectiveness of automated machine learning solutions. In the paper, we propose a modular approach that can be used to increase the quality of evolutionary optimization for modelling pipelines with a graph-based structure. It consists of several stages - parallelization, caching and evaluation. Heterogeneous and remote resources can be involved in the evaluation stage. The conducted experiments confirm the correctness and effectiveness of the proposed approach. The implemented algorithms are available as a part of the open-sour…
The graph partitioning problem (GPP) is a representative combinatorial optimization problem which is NP-hard. Currently, various approaches to solve GPP have been introduced. Among these, the GPP solution using evolutionary computation (EC) is an effective approach. There has not been any survey on the research applying EC to GPP since 2011. In this survey, we introduce various attempts to apply EC to GPP made in the recent seven years.
Eindhoven University of Technology（埃因霍温理工大学）；Nobleo Manufacturing（Nobleo制造公司）
Multi-objective evolutionary algorithms (MOEAs) are widely used to solve multi-objective optimization problems. The algorithms rely on setting appropriate parameters to find good solutions. However, this parameter tuning could be very computationally expensive in solving non-trial (combinatorial) optimization problems. This paper proposes a framework that integrates MOEAs with adaptive parameter control using Deep Reinforcement Learning (DRL). The DRL policy is trained to adaptively set the values that dictate the intensity and probability of mutation for solutions during optimization. We test…
University of Southampton（南安普顿大学）；The Alan Turing Institute（阿兰·图灵研究所）
Evolutionary Computation is a group of biologically inspired algorithms used to solve complex optimisation problems. It can be split into Evolutionary Algorithms, which take inspiration from genetic inheritance, and Swarm Intelligence algorithms, that take inspiration from cultural inheritance. However, recent developments have focused on computational or mathematical adaptions, leaving their biological roots behind. This has left much of the modern evolutionary literature relatively unexplored. To understand which evolutionary mechanisms have been considered, and which have been overlooked, t…
The cryptanalysis of various cipher problems can be formulated as NP-Hard combinatorial problem. Solving such problems requires time and/or memory requirement which increases with the size of the problem. Techniques for solving combinatorial problems fall into two broad groups - exact algorithms and Evolutionary Computation algorithms. An exact algorithms guarantees that the optimal solution to the problem will be found. The exact algorithms like branch and bound, simplex method, brute force etc methodology is very inefficient for solving combinatorial problem because of their prohibitive comp…
Imperial College London（帝国理工学院）；New York University（纽约大学）
In January 2019, DeepMind revealed AlphaStar to the world-the first artificial intelligence (AI) system to beat a professional player at the game of StarCraft II-representing a milestone in the progress of AI. AlphaStar draws on many areas of AI research, including deep learning, reinforcement learning, game theory, and evolutionary computation (EC). In this paper we analyze AlphaStar primarily through the lens of EC, presenting a new look at the system and relating it to many concepts in the field. We highlight some of its most interesting aspects-the use of Lamarckian evolution, competitive …
In this paper, we review the state-of-the-art results in evolutionary computation and observe that we do not evolve non trivial software from scratch and with no human intervention. A number of possible explanations are considered, but we conclude that computational complexity of the problem prevents it from being solved as currently attempted. A detailed analysis of necessary and available computational resources is provided to support our findings.
Algorithm::Evolutionary (A::E from now on) was introduced in 2002, after a talk in YAPC::EU in Munich. 7 years later, A::E is in its 0.67 version (past its "number of the beast" 0.666), and has been used extensively, to the point of being the foundation of much of the (computer) science being done by our research group (and, admittedly, not many others). All is not done, however; now A::E is being integrated with POE so that evolutionary algorithms (EAs) can be combined with all kinds of servers and used in client, servers, and anything in between. In this companion to the talk I will explain …
Over the past 30 years many researchers in the field of evolutionary computation have put a lot of effort to introduce various approaches for solving hard problems. Most of these problems have been inspired by major industries so that solving them, by providing either optimal or near optimal solution, was of major significance. Indeed, this was a very promising trajectory as advances in these problem-solving approaches could result in adding values to major industries. In this paper we revisit this trajectory to find out whether the attempts that started three decades ago are still aligned wit…
A common claim of evolutionary computation methods is that they can achieve good results without the need for human intervention. However, one criticism of this is that there are still hyperparameters which must be tuned in order to achieve good performance. In this work, we propose a near "parameter-free" genetic programming approach, which adapts the hyperparameter values throughout evolution without ever needing to be specified manually. We apply this to the area of automated machine learning (by extending TPOT), to produce pipelines which can effectively be claimed to be free from human in…
Now-a-days, it is important to find out solutions of Multi-Objective Optimization Problems (MOPs). Evolutionary Strategy helps to solve such real world problems efficiently and quickly. But sequential Evolutionary Algorithms (EAs) require an enormous computation power to solve such problems and it takes much time to solve large problems. To enhance the performance for solving this type of problems, this paper presents a new Distributed Novel Evolutionary Strategy Algorithm (DNESA) for Multi-Objective Optimization. The proposed DNESA applies the divide-and-conquer approach to decompose populati…
The challenge of ad-hoc computing is to find the way of taking advantage of spare cycles in an efficient way that takes into account all capabilities of the devices and interconnections available to them. In this paper we explore distributed evolutionary computation based on the Ruby on Rails framework, which overlays a Model-View-Controller on evolutionary computation. It allows anybody with a web browser (that is, mostly everybody connected to the Internet) to participate in an evolutionary computation experiment. Using a straightforward farming model, we consider different factors, such as …
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Canadian Institute for Advanced Research (CIFAR)（加拿大高等研究院）
Natural evolutionary strategies (NES) are a family of gradient-free black-box optimization algorithms. This study illustrates their use for the optimization of randomly-initialized parametrized quantum circuits (PQCs) in the region of vanishing gradients. We show that using the NES gradient estimator the exponential decrease in variance can be alleviated. We implement two specific approaches, the exponential and separable natural evolutionary strategies, for parameter optimization of PQCs and compare them against standard gradient descent. We apply them to two different problems of ground stat…
Barcelona Supercomputing Center（巴塞罗那超级计算中心）；IBM Research - Zurich（IBM苏黎世研究院）
In this work, we leverage ensemble learning as a tool for the creation of faster, smaller, and more accurate deep learning models. We demonstrate that we can jointly optimize for accuracy, inference time, and the number of parameters by combining DNN classifiers. To achieve this, we combine multiple ensemble strategies: bagging, boosting, and an ordered chain of classifiers. To reduce the number of DNN ensemble evaluations during the search, we propose EARN, an evolutionary approach that optimizes the ensemble according to three objectives regarding the constraints specified by the user. We ru…
One of the main properties of biological systems is modularity, which manifests itself at all levels of their organization, starting with the level of molecular genetics, ending with the level of whole organisms and their communities. In a simplified form, these basic principles were transferred from the genetics of populations to the field of evolutionary computations, in order to solve applied optimization problems. Over almost half a century of development in this field of computer science, considerable practical experience has been gained and interesting theoretical results have been obtai…
This paper introduces a novel idea for representation of individuals using quaternions in swarm intelligence and evolutionary algorithms. Quaternions are a number system, which extends complex numbers. They are successfully applied to problems of theoretical physics and to those areas needing fast rotation calculations. We propose the application of quaternions in optimization, more precisely, we have been using quaternions for representation of individuals in Bat algorithm. The preliminary results of our experiments when optimizing a test-suite consisting of ten standard functions showed that…
We report a summary of our interdisciplinary research project "Evolutionary Perspective on Collective Decision Making" that was conducted through close collaboration between computational, organizational and social scientists at Binghamton University. We redefined collective human decision making and creativity as evolution of ecologies of ideas, where populations of ideas evolve via continual applications of evolutionary operators such as reproduction, recombination, mutation, selection, and migration of ideas, each conducted by participating humans. Based on this evolutionary perspective, we…
Brainstorming is an effective technique for offline ideation although the number of participants able to join an ideation session and suggest ideas is limited. To increase the diversity and quality of the ideas suggested, many participants with various backgrounds should be able to join the session. We have devised an evolutionary computation-assisted brainwriting method for large-scale online ideation. In this method, participants not only suggest ideas but also evaluate ideas previously suggested by other participants. The evaluation results are used in the evolutionary computation to identi…
University of Málaga（马拉加大学）；University of Warwick（华威大学）；University of Coimbra（科英布拉大学）
Experimental studies are prevalent in Evolutionary Computation (EC), and concerns about the reproducibility and replicability of such studies have increased in recent times, reflecting similar concerns in other scientific fields. In this article, we discuss, within the context of EC, the different types of reproducibility and suggest a classification that refines the badge system of the Association of Computing Machinery (ACM) adopted by ACM Transactions on Evolutionary Learning and Optimization (https://dlnext.acm.org/journal/telo). We identify cultural and technical obstacles to reproducibil…
Online conference proceedings for the IWDECIE workshop, taking place in New Orleans on June 5th, 2011. The workshop focuses on non-conventional implementations of bioinspired algorithms and its conceptual implications.
The large number of exact fitness function evaluations makes evolutionary algorithms to have computational cost. In some real-world problems, reducing number of these evaluations is much more valuable even by increasing computational complexity and spending more time. To fulfill this target, we introduce an effective factor, in spite of applied factor in Adaptive Fuzzy Fitness Granulation with Non-dominated Sorting Genetic Algorithm-II, to filter out worthless individuals more precisely. Our proposed approach is compared with respect to Adaptive Fuzzy Fitness Granulation with Non-dominated Sor…
Guangzhou Institute of Technology, Xidian University（西安电子科技大学广州研究院）；South China University of Technology（华南理工大学）；Singapore Management University（新加坡管理大学）；Fuzhou University（福州大学）；Peng Cheng Laboratory（鹏城实验室）；Xidian University（西安电子科技大学）；Zhejiang Normal University（浙江师范大学）；Hanyang University（汉阳大学）
Most experimental studies initialize the population of evolutionary algorithms with random genotypes. In practice, however, optimizers are typically seeded with good candidate solutions either previously known or created according to some problem-specific method. This "seeding" has been studied extensively for single-objective problems. For multi-objective problems, however, very little literature is available on the approaches to seeding and their individual benefits and disadvantages. In this article, we are trying to narrow this gap via a comprehensive computational study on common real-val…
Institute of Automation and Computer Science, Brno University of Technology（布尔诺理工大学自动化与计算机科学学院）
The center-bias (or zero-bias) operator has recently been identified as one of the problems plaguing the benchmarking of evolutionary computation methods. This operator lets the methods that utilize it easily optimize functions that have their respective optima in the center of the feasible set. In this paper, we describe a simple procedure that can be used to identify methods that incorporate a center-bias operator and use it to investigate 90 evolutionary computation methods that were published between 1987 and 2022. We show that more than half (47 out of the 90) of the considered methods ha…
This erratum points out an error in the simplified drift theorem (SDT) [Algorithmica 59(3), 369-386, 2011]. It is also shown that a minor modification of one of its conditions is sufficient to establish a valid result. In many respects, the new theorem is more general than before. We no longer assume a Markov process nor a finite search space. Furthermore, the proof of the theorem is more compact than the previous ones. Finally, previous applications of the SDT are revisited. It turns out that all of these either meet the modified condition directly or by means of few additional arguments.
Massachusetts Institute of Technology, CSAIL（麻省理工学院计算机科学与人工智能实验室）
Generative Adversarial Networks (GANs) are popular tools for generative modeling. The dynamics of their adversarial learning give rise to convergence pathologies during training such as mode and discriminator collapse. In machine learning, ensembles of predictors demonstrate better results than a single predictor for many tasks. In this study, we apply two evolutionary algorithms (EAs) to create ensembles to re-purpose generative models, i.e., given a set of heterogeneous generators that were optimized for one objective (e.g., minimize Frechet Inception Distance), create ensembles of them for …
It is generally accepted that "diversity" is associated with success in evolutionary algorithms. However, diversity is a broad concept that can be measured and defined in a multitude of ways. To date, most evolutionary computation research has measured diversity using the richness and/or evenness of a particular genotypic or phenotypic property. While these metrics are informative, we hypothesize that other diversity metrics are more strongly predictive of success. Phylogenetic diversity metrics are a class of metrics popularly used in biology, which take into account the evolutionary history …
In modern data science, it is often not enough to obtain only a data-driven model with a good prediction quality. On the contrary, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. Such questions are unified under machine learning interpretability questions, which could be considered one of the area's raising topics. In the paper, we use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm's desired properties. It means that whereas one of the apparent objectives is pre…
Customisation in food properties is a challenging task involving optimisation of the production process with the demand to support computational creativity which is geared towards ensuring the presence of alternatives. This paper addresses the personalisation of beer properties in the specific case of craft beers where the production process is more flexible. We investigate the problem by using three swarm intelligence and evolutionary computation techniques that enable brewers to map physico-chemical properties to target organoleptic properties to design a specific brew. While there are sever…
The fusion of the multi-agent paradigm with evolutionary computation yielded promising results in many optimization problems. Evolutionary multi-agent system (EMAS) are more similar to biological evolution than classical evolutionary algorithms. However, technological limitations prevented the use of fully asynchronous agents in previous EMAS implementations. In this paper we present a new algorithm for agent-based evolutionary computations. The individuals are represented as fully autonomous and asynchronous agents. An efficient implementation of this algorithm was possible through the use of…
The performance of a deep neural network is heavily dependent on its architecture and various neural architecture search strategies have been developed for automated network architecture design. Recently, evolutionary neural architecture search (ENAS) has received increasing attention due to the attractive global optimization capability of evolutionary algorithms. However, ENAS suffers from extremely high computation costs because a large number of performance evaluations is usually required in evolutionary optimization and training deep neural networks is itself computationally very intensive…
With the volume of manuscripts submitted for publication growing every year, the deficiencies of peer review (e.g. long review times) are becoming more apparent. Editorial strategies, sets of guidelines designed to speed up the process and reduce editors workloads, are treated as trade secrets by publishing houses and are not shared publicly. To improve the effectiveness of their strategies, editors in small publishing groups are faced with undertaking an iterative trial-and-error approach. We show that Cartesian Genetic Programming, a nature-inspired evolutionary algorithm, can dramatically i…
Zhengzhou University（郑州大学）；Victoria University of Wellington（惠灵顿维多利亚大学）；University of Granada（格拉纳达大学）；Andalusian Research Institute in Artificial Intelligence（安达卢西亚人工智能研究所）；Panacea Cooperative Research S.Coop.（Panacea合作研究合作社）；University of Parma（帕尔马大学）
Computer vision (CV) is a big and important field in artificial intelligence covering a wide range of applications. Image analysis is a major task in CV aiming to extract, analyse and understand the visual content of images. However, image-related tasks are very challenging due to many factors, e.g., high variations across images, high dimensionality, domain expertise requirement, and image distortions. Evolutionary computation (EC) approaches have been widely used for image analysis with significant achievement. However, there is no comprehensive survey of existing EC approaches to image anal…
We introduce a novel evolutionary algorithm (EA) with a semantic network-based representation. For enabling this, we establish new formulations of EA variation operators, crossover and mutation, that we adapt to work on semantic networks. The algorithm employs commonsense reasoning to ensure all operations preserve the meaningfulness of the networks, using ConceptNet and WordNet knowledge bases. The algorithm can be interpreted as a novel memetic algorithm (MA), given that (1) individuals represent pieces of information that undergo evolution, as in the original sense of memetics as it was int…
