Evolutionary Design of Numerical Methods: Generating Finite Difference and Integration Schemes by Differential Evolution
National Research University of Information Technologies, Mechanics and Optics（信息技术、力学和光学国立研究大学）；University of Amsterdam（阿姆斯特丹大学）；Deltares；Nanyang Technological University（南洋理工大学）
C. D. Erdbrink, V. V. Krzhizhanovskaya, P. M. A. Sloot
Classical and new numerical schemes are generated using evolutionary computing. Differential Evolution is used to find the coefficients of finite difference approximations of function derivatives, and of single and multi-step integration methods. The coefficients are reverse engineered based on samples from a target function and its derivative used for training. The Runge-Kutta schemes are trained using the order condition equations. An appealing feature of the evolutionary method is the low number of model parameters. The population size, termination criterion and number of training points are determined in a sensitivity analysis. Computational results show good agreement between evolved and analytical coefficients. In particular, a new fifth-order Runge-Kutta scheme is computed which adheres to the order conditions with a sum of absolute errors of order 10^-14. Execution of the evolved schemes proved the intended orders of accuracy. The outcome of this study is valuable for future developments in the design of complex numerical methods that are out of reach by conventional means.
Barcelona Supercomputing Center（巴塞罗那超级计算中心）；IBM Research - Zurich（IBM苏黎世研究院）
In this work, we leverage ensemble learning as a tool for the creation of faster, smaller, and more accurate deep learning models. We demonstrate that we can jointly optimize for accuracy, inference time, and the number of parameters by combining DNN classifiers. To achieve this, we combine multiple ensemble strategies: bagging, boosting, and an ordered chain of classifiers. To reduce the number of DNN ensemble evaluations during the search, we propose EARN, an evolutionary approach that optimizes the ensemble according to three objectives regarding the constraints specified by the user. We ru…
Khulna University of Engineering & Technology（库尔纳工程技术大学）
Fuzzy systems may be considered as knowledge-based systems that incorporates human knowledge into their knowledge base through fuzzy rules and fuzzy membership functions. The intent of this study is to present a fuzzy knowledge integration framework using a Novel Evolutionary Strategy (NES), which can simultaneously integrate multiple fuzzy rule sets and their membership function sets. The proposed approach consists of two phases: fuzzy knowledge encoding and fuzzy knowledge integration. Four application domains, the hepatitis diagnosis, the sugarcane breeding prediction, Iris plants classific…
Deep neural networks (DNNs) are sensitive to adversarial data in a variety of scenarios, including the black-box scenario, where the attacker is only allowed to query the trained model and receive an output. Existing black-box methods for creating adversarial instances are costly, often using gradient estimation or training a replacement network. This paper introduces \textbf{Qu}ery-Efficient \textbf{E}volutiona\textbf{ry} \textbf{Attack}, \textit{QuEry Attack}, an untargeted, score-based, black-box attack. QuEry Attack is based on a novel objective function that can be used in gradient-free o…
Generating various strategies for a given task is challenging. However, it has already proven to bring many assets to the main learning process, such as improved behavior exploration. With the growth in the interest of heterogeneity in solution in evolutionary computation and reinforcement learning, many promising approaches have emerged. To better understand how one guides multiple policies toward distinct strategies and benefit from diversity, we need to analyze further the influence of the reward signal modulation and other evolutionary mechanisms on the obtained behaviors. To that effect, …
A master face is a face image that passes face-based identity-authentication for a large portion of the population. These faces can be used to impersonate, with a high probability of success, any user, without having access to any user-information. We optimize these faces, by using an evolutionary algorithm in the latent embedding space of the StyleGAN face generator. Multiple evolutionary strategies are compared, and we propose a novel approach that employs a neural network in order to direct the search in the direction of promising samples, without adding fitness evaluations. The results we …
A software program which aims to provide an exploration capability over the Search Space of potential drug molecules. The program explores the search space by generating random molecules, determining their fitness and then breeding a new generation from the fittest individuals. The search space, in theory any combination of any elements in any order, is constrained by the use of a subset of elements and a list of fragments, molecular parts that are known to be useful in drug development. The resultant molecules from each generation are stored in a searchable database, so that the user can brow…
Universidade da Beira Interior（贝拉内格罗大学）；Instituto de Telecomunicações（电信研究所）；DeepNeuronic（DeepNeuronic公司）
Neural Architecture Search (NAS) methods have been successfully applied to image tasks with excellent results. However, NAS methods are often complex and tend to converge to local minima as soon as generated architectures seem to yield good results. This paper proposes GEA, a novel approach for guided NAS. GEA guides the evolution by exploring the search space by generating and evaluating several architectures in each generation at initialisation stage using a zero-proxy estimator, where only the highest-scoring architecture is trained and kept for the next generation. Subsequently, GEA contin…
Southern University of Science and Technology（南方科技大学）；City University of Hong Kong（香港城市大学）；University of Surrey（萨里大学）
Recently, increasing works have proposed to drive evolutionary algorithms using machine learning models. Usually, the performance of such model based evolutionary algorithms is highly dependent on the training qualities of the adopted models. Since it usually requires a certain amount of data (i.e. the candidate solutions generated by the algorithms) for model training, the performance deteriorates rapidly with the increase of the problem scales, due to the curse of dimensionality. To address this issue, we propose a multi-objective evolutionary algorithm driven by the generative adversarial n…
Most existing studies on evolutionary multi-objective optimization focus on approximating the whole Pareto-optimal front. Nevertheless, rather than the whole front, which demands for too many points (especially in a high-dimensional space), the decision maker might only interest in a partial region, called the region of interest. In this case, solutions outside this region can be noisy to the decision making procedure. Even worse, there is no guarantee that we can find the preferred solutions when tackling problems with complicated properties or a large number of objectives. In this paper, we …
Recently, more and more works have proposed to drive evolutionary algorithms using machine learning models.Usually, the performance of such model based evolutionary algorithms is highly dependent on the training qualities of the adopted models.Since it usually requires a certain amount of data (i.e. the candidate solutions generated by the algorithms) for model training, the performance deteriorates rapidly with the increase of the problem scales, due to the curse of dimensionality.To address this issue, we propose a multi-objective evolutionary algorithm driven by the generative adversarial n…
Southern University of Science and Technology（南方科技大学）；NetEase Games AI Lab（网易游戏人工智能实验室）
Despite the emerging progress of integrating evolutionary computation into reinforcement learning, the absence of a high-performance platform endowing composability and massive parallelism causes non-trivial difficulties for research and applications related to asynchronous commercial games. Here we introduce Lamarckian - an open-source platform featuring support for evolutionary reinforcement learning scalable to distributed computing resources. To improve the training speed and data efficiency, Lamarckian adopts optimized communication methods and an asynchronous evolutionary reinforcement l…
Generating multiple categories of texts is a challenging task and draws more and more attention. Since generative adversarial nets (GANs) have shown competitive results on general text generation, they are extended for category text generation in some previous works. However, the complicated model structures and learning strategies limit their performance and exacerbate the training instability. This paper proposes a category-aware GAN (CatGAN) which consists of an efficient category-aware model for category text generation and a hierarchical evolutionary learning algorithm for training our mo…
In this paper we propose a novel method for learning how algorithms perform. Classically, algorithms are compared on a finite number of existing (or newly simulated) benchmark datasets based on some fixed metrics. The algorithm(s) with the smallest value of this metric are chosen to be the `best performing'. We offer a new approach to flip this paradigm. We instead aim to gain a richer picture of the performance of an algorithm by generating artificial data through genetic evolution, the purpose of which is to create populations of datasets for which a particular algorithm performs well on a g…
A master face is a face image that passes face-based identity authentication for a high percentage of the population. These faces can be used to impersonate, with a high probability of success, any user, without having access to any user information. We optimize these faces for 2D and 3D face verification models, by using an evolutionary algorithm in the latent embedding space of the StyleGAN face generator. For 2D face verification, multiple evolutionary strategies are compared, and we propose a novel approach that employs a neural network to direct the search toward promising samples, withou…
Building self-adaptive and self-organizing (SASO) systems is a challenging problem, in part because SASO principles are not yet well understood and few platforms exist for exploring them. Cellular automata (CA) are a well-studied approach to exploring the principles underlying self-organization. A CA comprises a lattice of cells whose states change over time based on a discrete update function. One challenge to developing CA is that the relationship of an update function, which describes the local behavior of each cell, to the global behavior of the entire CA is often unclear. As a result, man…
Sookmyung Women’s University（淑明女子大学）；Korea Advanced Institute of Science and Technology（韩国科学技术院）
Deep learning has recently been applied to various research areas of design optimization. This study presents the need and effectiveness of adopting deep learning for generative design (or design exploration) research area. This work proposes an artificial intelligent (AI)-based design automation framework that is capable of generating numerous design options which are not only aesthetic but also optimized for engineering performance. The proposed framework integrates topology optimization and deep generative models (e.g., generative adversarial networks (GANs)) in an iterative manner to explo…
