Optimization of Evolutionary Neural Networks Using Hybrid Learning Algorithms
Ajith Abraham
发表信息：
IEEE International Joint Conference on Neural Networks (IJCNN'02), 2002 IEEE World Congress on Computational Intelligence, Hawaii, ISBN 0780372786, IEEE Press, Volume 3, pp. 2797-2802, 2002
Evolutionary artificial neural networks (EANNs) refer to a special class of artificial neural networks (ANNs) in which evolution is another fundamental form of adaptation in addition to learning. Evolutionary algorithms are used to adapt the connection weights, network architecture and learning algorithms according to the problem environment. Even though evolutionary algorithms are well known as efficient global search algorithms, very often they miss the best local solutions in the complex solution space. In this paper, we propose a hybrid meta-heuristic learning approach combining evolutionary learning and local search methods (using 1st and 2nd order error information) to improve the learning and faster convergence obtained using a direct evolutionary approach. The proposed technique is tested on three different chaotic time series and the test results are compared with some popular neuro-fuzzy systems and a recently developed cutting angle method of global optimization. Empirical results reveal that the proposed technique is efficient in spite of the computational complexity.
Several adaptation techniques have been investigated to optimize fuzzy inference systems. Neural network learning algorithms have been used to determine the parameters of fuzzy inference system. Such models are often called as integrated neuro-fuzzy models. In an integrated neuro-fuzzy model there is no guarantee that the neural network learning algorithm converges and the tuning of fuzzy inference system will be successful. Success of evolutionary search procedures for optimization of fuzzy inference system is well proven and established in many application areas. In this paper, we will explo…
University of Georgia（佐治亚大学）；Florida International University（佛罗里达国际大学）
In the past 30 years, scientists have searched nature, including animals and insects, and biology in order to discover, understand, and model solutions for solving large-scale science challenges. The study of bionics reveals that how the biological structures, functions found in nature have improved our modern technologies. In this study, we present our discovery of evolutionary and nature-inspired algorithms applications in Data Science and Data Analytics in three main topics of pre-processing, supervised algorithms, and unsupervised algorithms. Among all applications, in this study, we aim t…
This paper presents a new multi-objective hybrid model that makes cooperation between the strength of research of neighborhood methods presented by the tabu search (TS) and the important exploration capacity of evolutionary algorithm. This model was implemented and tested in benchmark functions (ZDT1, ZDT2, and ZDT3), using a network of computers.
Artificial intelligence is one of the drivers of modern technological development. The current approach to the development of intelligent systems is data-centric. It has several limitations: it is fundamentally impossible to collect data for modeling complex objects and processes; training neural networks requires huge computational and energy resources; solutions are not explainable. The article discusses an alternative approach to the development of artificial intelligence systems based on human-machine hybridization and their co-evolution.
Artificial Intelligence Research Institute (IIIA-CSIC)（人工智能研究所 (IIIA-CSIC)）；Universitat Politècnica de Catalunya (BarcelonaTech)（加泰罗尼亚理工大学 (BarcelonaTech)）
The longest arc-preserving common subsequence problem is an NP-hard combinatorial optimization problem from the field of computational biology. This problem finds applications, in particular, in the comparison of arc-annotated Ribonucleic acid (RNA) sequences. In this work we propose a simple, hybrid evolutionary algorithm to tackle this problem. The most important feature of this algorithm concerns a crossover operator based on solution merging. In solution merging, two or more solutions to the problem are merged, and an exact technique is used to find the best solution within this union. It …
Functional brain imaging is a source of spatio-temporal data mining problems. A new framework hybridizing multi-objective and multi-modal optimization is proposed to formalize these data mining problems, and addressed through Evolutionary Computation (EC). The merits of EC for spatio-temporal data mining are demonstrated as the approach facilitates the modelling of the experts' requirements, and flexibly accommodates their changing goals.
Virginia Tech（弗吉尼亚理工学院）；US Army Research Laboratory（美国陆军研究实验室）；US Army, ERDC-GRL（美国陆军ERDC-GRL）
In this paper, we propose a surrogate-assisted evolutionary algorithm (EA) for hyperparameter optimization of machine learning (ML) models. The proposed STEADE model initially estimates the objective function landscape using RadialBasis Function interpolation, and then transfers the knowledge to an EA technique called Differential Evolution that is used to evolve new solutions guided by a Bayesian optimization framework. We empirically evaluate our model on the hyperparameter optimization problems as a part of the black box optimization challenge at NeurIPS 2020 and demonstrate the improvement…
Making high-quality decisions in strategic spatial planning is heavily dependent on extracting knowledge from vast amounts of data. Although many decision-making problems like developing urban areas require such perception and reasoning, existing methods in this field usually neglect the deep knowledge mined from geographic databases and are based on pure statistical methods. Due to the large volume of data gathered in spatial databases, and the uncertainty of spatial objects, mining association rules for high-level knowledge representation is a challenging task. Few algorithms manage geograph…
Train timetabling is a difficult and very tightly constrained combinatorial problem that deals with the construction of train schedules. We focus on the particular problem of local reconstruction of the schedule following a small perturbation, seeking minimisation of the total accumulated delay by adapting times of departure and arrival for each train and allocation of resources (tracks, routing nodes, etc.). We describe a permutation-based evolutionary algorithm that relies on a semi-greedy heuristic to gradually reconstruct the schedule by inserting trains one after the other following the p…
National University of Singapore（新加坡国立大学）；Seagate（希捷科技）
This paper presents an evolutionary algorithm with a new goal-sequence domination scheme for better decision support in multi-objective optimization. The approach allows the inclusion of advanced hard/soft priority and constraint information on each objective component, and is capable of incorporating multiple specifications with overlapping or non-overlapping objective functions via logical 'OR' and 'AND' connectives to drive the search towards multiple regions of trade-off. In addition, we propose a dynamic sharing scheme that is simple and adaptively estimated according to the on-line popul…
Multi-task learning (MTL), which aims to improve performance by learning multiple tasks simultaneously, inherently presents an optimization challenge due to multiple objectives. Hence, multi-objective optimization (MOO) approaches have been proposed for multitasking problems. Recent MOO methods approximate multiple optimal solutions (Pareto front) with a single unified model, which is collectively referred to as Pareto front learning (PFL). In this paper, we show that PFL can be re-formulated into another MOO problem with multiple objectives, each of which corresponds to different preference w…
Southern University of Science and Technology（南方科技大学）
The reliable facility location problem (RFLP) is an important research topic of operational research and plays a vital role in the decision-making and management of modern supply chain and logistics. Through solving RFLP, the decision-maker can obtain reliable location decisions under the risk of facilities' disruptions or failures. In this paper, we propose a novel model for the RFLP. Instead of assuming allocating a fixed number of facilities to each customer as in the existing works, we set the number of allocated facilities as an independent variable in our proposed model, which makes our …
Evolutionary processes proved very useful for solving optimization problems. In this work, we build a formalization of the notion of cooperation and competition of multiple systems working toward a common optimization goal of the population using evolutionary computation techniques. It is justified that evolutionary algorithms are more expressive than conventional recursive algorithms. Three subclasses of evolutionary algorithms are proposed here: bounded finite, unbounded finite and infinite types. Some results on completeness, optimality and search decidability for the above classes are pres…
Institute for Development and Research in Banking Technology（银行技术发展与研究院）；University of Hyderabad（海得拉巴大学）
In management, business, economics, science, engineering, and research domains, Large Scale Global Optimization (LSGO) plays a predominant and vital role. Though LSGO is applied in many of the application domains, it is a very troublesome and a perverse task. The Congress on Evolutionary Computation (CEC) began an LSGO competition to come up with algorithms with a bunch of standard benchmark unconstrained LSGO functions. Therefore, in this paper, we propose a hybrid meta-heuristic algorithm, which combines an Improved and Modified Harmony Search (IMHS), along with a Modified Differential Evolu…
Beijing Institute of Technology（北京理工大学）；Victoria University of Wellington（惠灵顿维多利亚大学）
An emerging optimisation problem from the real-world applications, named the multi-point dynamic aggregation (MPDA) problem, has become one of the active research topics of the multi-robot system. This paper focuses on a multi-objective MPDA problem which is to design an execution plan of the robots to minimise the number of robots and the maximal completion time of all the tasks. The strongly-coupled relationships among robots and tasks, the redundancy of the MPDA encoding, and the variable-size decision space of the MO-MPDA problem posed extra challenges for addressing the problem effectivel…
Novosibirsk State Technical University（新西伯利亚国立技术大学）
The article presents a study of the Particle Swarm optimization method for scheduling problem. To improve the method's performance a restriction of particles' velocity and an evolutionary meta-optimization were realized. The approach proposed uses the Genetic algorithms for selection of the parameters of Particle Swarm optimization. Experiments were carried out on test tasks of the job-shop scheduling problem. This research proves the applicability of the approach and shows the importance of tuning the behavioral parameters of the swarm intelligence methods to achieve a high performance.
University of Science and Technology of China（中国科学技术大学）；Loongson Technologies Corporation Limited（龙芯技术有限公司）；University of Birmingham（伯明翰大学）
Mutation has traditionally been regarded as an important operator in evolutionary algorithms. In particular, there have been many experimental studies which showed the effectiveness of adapting mutation rates for various static optimization problems. Given the perceived effectiveness of adaptive and self-adaptive mutation for static optimization problems, there have been speculations that adaptive and self-adaptive mutation can benefit dynamic optimization problems even more since adaptation and self-adaptation are capable of following a dynamic environment. However, few theoretical results ar…
East China University of Science and Technology（华东理工大学）；University of Surrey（萨里大学）
Data-driven optimization has found many successful applications in the real world and received increased attention in the field of evolutionary optimization. Most existing algorithms assume that the data used for optimization is always available on a central server for construction of surrogates. This assumption, however, may fail to hold when the data must be collected in a distributed way and is subject to privacy restrictions. This paper aims to propose a federated data-driven evolutionary multi-/many-objective optimization algorithm. To this end, we leverage federated learning for surrogat…
In this article a tool for the analysis of population-based EAs is used to derive asymptotic upper bounds on the optimization time of the algorithm solving Royal Roads problem, a test function with plateaus of fitness. In addition to this, limiting distribution of a certain subset of the population is approximated.
Shenzhen University（深圳大学）；The Hong Kong Polytechnic University（香港理工大学）
Recent decades have witnessed great advancements in multiobjective evolutionary algorithms (MOEAs) for multiobjective optimization problems (MOPs). However, these progressively improved MOEAs have not necessarily been equipped with scalable and learnable problem-solving strategies for new and grand challenges brought by the scaling-up MOPs with continuously increasing complexity from diverse aspects, mainly including expensive cost of function evaluations, many objectives, large-scale search space, time-varying environments, and multi-task. Under different scenarios, divergent thinking is requ…
Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学院）；Montreal Institute of Learning Algorithms (MILA)（蒙特利尔学习算法研究所）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）
EvoSampling: A Granular Ball-based Evolutionary Hybrid Sampling with Knowledge Transfer for Imbalanced Learning
School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）；Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University of Technology), Ministry of Education（社会计算与认知智能重点实验室（大连理工大学））；School of Engineering and Computer Science, Victoria University of Wellington（惠灵顿维多利亚大学工程与计算机科学学院）；Department of Computer Science, Hong Kong Baptist University（香港浸会大学计算机科学系）
Real world problems always have different multiple solutions. For instance, optical engineers need to tune the recording parameters to get as many optimal solutions as possible for multiple trials in the varied-line-spacing holographic grating design problem. Unfortunately, most traditional optimization techniques focus on solving for a single optimal solution. They need to be applied several times; yet all solutions are not guaranteed to be found. Thus the multimodal optimization problem was proposed. In that problem, we are interested in not only a single optimal point, but also the others. …
CNRS（法国国家科学研究中心）；École Polytechnique（巴黎综合理工学院）；Institut Polytechnique de Paris（巴黎综合理工学院）；The University of Adelaide（阿德莱德大学）
The theory of evolutionary computation for discrete search spaces has made significant progress in the last ten years. This survey summarizes some of the most important recent results in this research area. It discusses fine-grained models of runtime analysis of evolutionary algorithms, highlights recent theoretical insights on parameter tuning and parameter control, and summarizes the latest advances for stochastic and dynamic problems. We regard how evolutionary algorithms optimize submodular functions and we give an overview over the large body of recent results on estimation of distributio…
Structure-Aware Cooperative Ensemble Evolutionary Optimization on Combinatorial Problems with Multimodal Large Language Models
School of Physical and Mathematical Sciences, Nanyang Technological University（南洋理工大学物理与数学科学学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
College of Systems Engineering, National University of Defense Technology（国防科技大学系统工程学院）；Hunan Key Laboratory of Multi-Energy System Intelligent Interconnection Technology (HKL-MESIT)（湖南省多能源系统智能互联技术重点实验室）
This paper contributes to the treatment of extensive constraints in evolutionary many-constraint optimization through consideration of the relationships between pair-wise constraints. In a conflicting relationship, the functional value of one constraint increases as the value in another constraint decreases. In a harmonious relationship, the improvement in one constraint is rewarded with simultaneous improvement in the other constraint. In an independent relationship, the adjustment to one constraint never affects the adjustment to the other. Based on the different features, methods for identi…
AGH University of Science and Technology（AGH科技大学）；Tomas Bata University in Zlín（兹林托马斯·巴塔大学）
Metaheuristics are universal optimization algorithms which should be used for solving difficult problems, unsolvable by classic approaches. In this paper we aim at constructing novel socio-cognitive metaheuristic based on castes, and apply several versions of this algorithm to optimization of time-delay system model. Besides giving the background and the details of the proposed algorithms we apply them to optimization of selected variants of the problem and discuss the results.
Guangzhou Institute of Technology, Xidian University（西安电子科技大学广州研究院）；South China University of Technology（华南理工大学）；Singapore Management University（新加坡管理大学）；Fuzhou University（福州大学）；Peng Cheng Laboratory（鹏城实验室）；Xidian University（西安电子科技大学）；Zhejiang Normal University（浙江师范大学）；Hanyang University（汉阳大学）
Ecole nationale des Sciences de l’Informatique (ENSI)（国立计算机科学学院（ENSI））；ISG de Tunis（突尼斯高等管理学院）；Université de Tunis（突尼斯大学）
We consider a multi-location inventory system where inventory choices at each location are centrally coordinated. Lateral transshipments are allowed as recourse actions within the same echelon in the inventory system to reduce costs and improve service level. However, this transshipment process usually causes undesirable lead times. In this paper, we propose a multiobjective model of the multi-location transshipment problem which addresses optimizing three conflicting objectives: (1) minimizing the aggregate expected cost, (2) maximizing the expected fill rate, and (3) minimizing the expected …
Generating various strategies for a given task is challenging. However, it has already proven to bring many assets to the main learning process, such as improved behavior exploration. With the growth in the interest of heterogeneity in solution in evolutionary computation and reinforcement learning, many promising approaches have emerged. To better understand how one guides multiple policies toward distinct strategies and benefit from diversity, we need to analyze further the influence of the reward signal modulation and other evolutionary mechanisms on the obtained behaviors. To that effect, …
Chongqing University（重庆大学）；Dalian University of Technology（大连理工大学）；City University of Hong Kong（香港城市大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）；Nanyang Technological University（南洋理工大学）
In recent years, to improve the evolutionary algorithms used to solve optimization problems involving a large number of decision variables, many attempts have been made to simplify the problem solution space of a given problem for the evolutionary search. In the literature, the existing approaches can generally be categorized as decomposition-based methods and dimension-reduction-based methods. The former decomposes a large-scale problem into several smaller subproblems, while the latter transforms the original high-dimensional solution space into a low-dimensional space. However, it is worth …
Sun Yat-sen University（中山大学）；Guangdong University of Finance and Economics（广东财经大学）
Learning-based heuristics for solving combinatorial optimization problems has recently attracted much academic attention. While most of the existing works only consider the single objective problem with simple constraints, many real-world problems have the multiobjective perspective and contain a rich set of constraints. This paper proposes a multiobjective deep reinforcement learning with evolutionary learning algorithm for a typical complex problem called the multiobjective vehicle routing problem with time windows (MO-VRPTW). In the proposed algorithm, the decomposition strategy is applied …
Queen Mary University of London（伦敦玛丽女王大学）；Microsoft Research Cambridge（微软剑桥研究院）
Game-playing Evolutionary Algorithms, specifically Rolling Horizon Evolutionary Algorithms, have recently managed to beat the state of the art in win rate across many video games. However, the best results in a game are highly dependent on the specific configuration of modifications and hybrids introduced over several papers, each adding additional parameters to the core algorithm. Further, the best previously published parameters have been found from only a few human-picked combinations, as the possibility space has grown beyond exhaustive search. This paper presents the state of the art in R…
Coronavirus Optimization Algorithm: A Success-History Adaptive Evolutionary Framework with Archive-Assisted Search and Stagnation Recovery for Global Optimization
TECNALIA, Basque Research and Technology Alliance (BRTA)（TECNALIA，巴斯克研究与技术联盟（BRTA））；Universidad de Cantabria（坎塔布里亚大学）；Toho University（东邦大学）；University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU））
The emerging research paradigm coined as multitasking optimization aims to solve multiple optimization tasks concurrently by means of a single search process. For this purpose, the exploitation of complementarities among the tasks to be solved is crucial, which is often achieved via the transfer of genetic material, thereby forging the Transfer Optimization field. In this context, Evolutionary Multitasking addresses this paradigm by resorting to concepts from Evolutionary Computation. Within this specific branch, approaches such as the Multifactorial Evolutionary Algorithm (MFEA) has lately ga…
