One of the most rising issues in recent machine learning research is One-Class Classification which considers data set composed of only one class and outliers. It is more reasonable than traditional Multi-Class Classification in dealing with some problematic data set or special cases. Generally, classification accuracy and interpretability for user are considered as trade-off in OCC methods. Classifier based on Hyper-Rectangle (H-RTGL) is a sort of classifier that can be a remedy for such trade-off and uses H-RTGL formulated by conjunction of geometric rules called interval. This interval can be basis of interpretability since it can be easily understood by user. However, existing H-RTGL based OCC classifiers have limitations that (i) most of them cannot reflect density of target class and (ii) that considering density has primitive interval generation method, and (iii) there exists no systematic procedure for hyperparameter of H-RTGL based OCC classifier, which influences classification performance of classifier. Based on these remarks, we suggest One-Class Hyper-Rectangle Descriptor based on density (1-HRD_d) with more elaborate interval generation method including parametric and nonparametric approaches. In addition, we designed Genetic Algorithm (GA) that consists of chromosome structure and genetic operators for systematic generation of 1-HRD_d by optimization of hyperparameter. Our work is validated through a numerical experiment using actual data set with comparison of existing OCC algorithms along with other H-RTGL based classifiers.
An approach to the acceleration of parametric weak classifier boosting is proposed. Weak classifier is called parametric if it has fixed number of parameters and, so, can be represented as a point into multidimensional space. Genetic algorithm is used instead of exhaustive search to learn parameters of such classifier. Proposed approach also takes cases when effective algorithm for learning some of the classifier parameters exists into account. Experiments confirm that such an approach can dramatically decrease classifier training time while keeping both training and test errors small.
Society has come to rely on algorithms like classifiers for important decision making, giving rise to the need for ethical guarantees such as fairness. Fairness is typically defined by asking that some statistic of a classifier be approximately equal over protected groups within a population. In this paper, current approaches to fairness are discussed and used to motivate algorithmic proposals that incorporate fairness into genetic programming for classification. We propose two ideas. The first is to incorporate a fairness objective into multi-objective optimization. The second is to adapt lex…
Using a Genetic Algorithm and Decision Tree Classifier, the features of the NSL-KDD dataset are reduced using combinatorial optimization to determine the minimum features required to accurately classify Denial of Service attacks within the NSL-KDD dataset.
Faculdade de Ciências da Universidade de Lisboa（里斯本大学科学学院）
Ensemble learning is a powerful paradigm that has been usedin the top state-of-the-art machine learning methods like Random Forestsand XGBoost. Inspired by the success of such methods, we have devel-oped a new Genetic Programming method called Ensemble GP. The evo-lutionary cycle of Ensemble GP follows the same steps as other GeneticProgramming systems, but with differences in the population structure,fitness evaluation and genetic operators. We have tested this method oneight binary classification problems, achieving results significantly betterthan standard GP, with much smaller models. Alth…
One of the most significant current discussions in the field of data mining is classifying imbalanced data. In recent years, several ways are proposed such as algorithm level (internal) approaches, data level (external) techniques, and cost-sensitive methods. Although extensive research has been carried out on imbalanced data classification, however, several unsolved challenges remain such as no attention to the importance of samples to balance, determine the appropriate number of classifiers, and no optimization of classifiers in the combination of classifiers. The purpose of this paper is to…
École de technologie supérieure (ÉTS), Université du Québec（魁北克大学高等技术学院）；Pontifícia Universidade Católica do Paraná (PUCPR)（巴拉那天主教大学）；Universidade Federal do Paraná (UFPR)（巴拉那联邦大学）
The Ensemble of Classifiers (EoC) has been shown to be effective in improving the performance of single classifiers by combining their outputs, and one of the most important properties involved in the selection of the best EoC from a pool of classifiers is considered to be classifier diversity. In general, classifier diversity does not occur randomly, but is generated systematically by various ensemble creation methods. By using diverse data subsets to train classifiers, these methods can create diverse classifiers for the EoC. In this work, we propose a scheme to measure data diversity direct…
NOVA IMS, Universidade Nova de Lisboa（里斯本新大学NOVA信息管理学院）；Università degli Studi di Milano-Bicocca（米兰比可卡大学）
The objective of this paper is to define an effective strategy for building an ensemble of Genetic Programming (GP) models. Ensemble methods are widely used in machine learning due to their features: they average out biases, they reduce the variance and they usually generalize better than single models. Despite these advantages, building ensemble of GP models is not a well-developed topic in the evolutionary computation community. To fill this gap, we propose a strategy that blends individuals produced by standard syntax-based GP and individuals produced by geometric semantic genetic programmi…
University of Waterloo（滑铁卢大学）；National Research Council of Canada（加拿大国家研究委员会）
In this paper, we use a variance-based genetic ensemble (VGE) of Neural Networks (NNs) to detect anomalies in the satellite's historical data. We use an efficient ensemble of the predictions from multiple Recurrent Neural Networks (RNNs) by leveraging each model's uncertainty level (variance). For prediction, each RNN is guided by a Genetic Algorithm (GA) which constructs the optimal structure for each RNN model. However, finding the model uncertainty level is challenging in many cases. Although the Bayesian NNs (BNNs)-based methods are popular for providing the confidence bound of the models,…
Credit card fraud detection based on machine learning has recently attracted considerable interest from the research community. One of the most important tasks in this area is the ability of classifiers to handle the imbalance in credit card data. In this scenario, classifiers tend to yield poor accuracy on the fraud class (minority class) despite realizing high overall accuracy. This is due to the influence of the majority class on traditional training criteria. In this paper, we aim to apply genetic programming to address this issue by adapting existing fitness functions. We examine two fitn…
Learning ensembles by bagging can substantially improve the generalization performance of low-bias, high-variance estimators, including those evolved by Genetic Programming (GP). To be efficient, modern GP algorithms for evolving (bagging) ensembles typically rely on several (often inter-connected) mechanisms and respective hyper-parameters, ultimately compromising ease of use. In this paper, we provide experimental evidence that such complexity might not be warranted. We show that minor changes to fitness evaluation and selection are sufficient to make a simple and otherwise-traditional GP al…
Recently we proposed a general, ensemble-based feature engineering wrapper (FEW) that was paired with a number of machine learning methods to solve regression problems. Here, we adapt FEW for supervised classification and perform a thorough analysis of fitness and survival methods within this framework. Our tests demonstrate that two fitness metrics, one introduced as an adaptation of the silhouette score, outperform the more commonly used Fisher criterion. We analyze survival methods and demonstrate that $ε$-lexicase survival works best across our test problems, followed by random survival wh…
AutoQML: Automatic Generation and Training of Robust Quantum-Inspired Classifiers by Using Genetic Algorithms on Grayscale Images
Consejo Superior de Investigaciones Científicas（西班牙国家研究委员会）；Centre for Automation and Robotics（自动化与机器人中心）；Universidad Politécnica de Madrid（马德里理工大学）；Instituto de Física Fundamental（基础物理研究所）
Florida State University（佛罗里达州立大学）；University of Chinese Academy of Sciences（中国科学院大学）
Deep learning, a rebranding of deep neural network research works, has achieved a remarkable success in recent years. With multiple hidden layers, deep learning models aim at computing the hierarchical feature representations of the observational data. Meanwhile, due to its severe disadvantages in data consumption, computational resources, parameter tuning costs and the lack of result explainability, deep learning has also suffered from lots of criticism. In this paper, we will introduce a new representation learning model, namely "Sample-Ensemble Genetic Evolutionary Network" (SEGEN), which c…
Machine learning algorithms are inherently multiobjective in nature, where approximation error minimization and model's complexity simplification are two conflicting objectives. We proposed a multiobjective genetic programming (MOGP) for creating a heterogeneous flexible neural tree (HFNT), tree-like flexible feedforward neural network model. The functional heterogeneity in neural tree nodes was introduced to capture a better insight of data during learning because each input in a dataset possess different features. MOGP guided an initial HFNT population towards Pareto-optimal solutions, where…
Epilepsy affects 50 million people worldwide and is one of the most common serious neurological disorders. Seizure detection and classification is a valuable tool for diagnosing and maintaining the condition. An automated classification algorithm will allow for accurate diagnosis. Utilising the Temple University Hospital (TUH) Seizure Corpus, six seizure types are compared; absence, complex partial, myoclonic, simple partial, tonic and tonic- clonic models. This study proposes a method that utilises unique features with a novel parallel classifier - Parallel Genetic Naive Bayes (NB) Seizure Cl…
Moscow Institute of Physics and Technology（莫斯科物理技术学院）
The problem of autonomous navigation is one of the basic problems for robotics. Although, in general, it may be challenging when an autonomous vehicle is placed into partially observable domain. In this paper we consider simplistic environment model and introduce a navigation algorithm based on Learning Classifier System.
Cracow University of Technology（克拉科夫科技大学）；Institute of Theoretical and Applied Informatics, Polish Academy of Sciences（波兰科学院理论与应用信息学研究所）
This study is focused on applying genetic algorithms (GA) to model and band selection in hyperspectral image classification. We use a forensic-inspired data set of seven hyperspectral images with blood and five visually similar substances to test GA-optimised classifiers in two scenarios: when the training and test data come from the same image and when they come from different images, which is a more challenging task due to significant spectra differences. In our experiments we compare GA with a classic model optimisation through grid search. Our results show that GA-based model optimisation …
Imbalanced datasets are ubiquitous. Classification performance on imbalanced datasets is generally poor for the minority class as the classifier cannot learn decision boundaries well. However, in sensitive applications like fraud detection, medical diagnosis, and spam identification, it is extremely important to classify the minority instances correctly. In this paper, we present a novel technique based on genetic algorithms, GenSample, for oversampling the minority class in imbalanced datasets. GenSample decides the rate of oversampling a minority example by taking into account the difficulty…
What is the value of an individual model in an ensemble of binary classifiers? We answer this question by introducing a class of transferable utility cooperative games called \textit{ensemble games}. In machine learning ensembles, pre-trained models cooperate to make classification decisions. To quantify the importance of models in these ensemble games, we define \textit{Troupe} -- an efficient algorithm which allocates payoffs based on approximate Shapley values of the classifiers. We argue that the Shapley value of models in these games is an effective decision metric for choosing a high per…
University of Kashmir（克什米尔大学）；South Valley University（南谷大学）
In multiclass deep network classifiers, the burden of classifying samples of different classes is put on a single classifier. As the result the optimum classification accuracy is not obtained. Also training times are large due to running the CNN training on single CPU/GPU. However it is known that using ensembles of classifiers increases the performance. Also, the training times can be reduced by running each member of the ensemble on a separate processor. Ensemble learning has been used in the past for traditional methods to a varying extent and is a hot topic. With the advent of deep learnin…
Multi-population Ensemble Genetic Programming via Cooperative Coevolution and Multi-view Learning for Classification
Faculty of Engineering & Information Technology, University of Technology Sydney（工程与信息科技学院，技术悉尼大学）；Department of Civil and Architectural Engineering, Sultan Qaboos University（土木与建筑工程系，苏丹·卡布斯大学）；University Research and Innovation Center (EKIK), Obuda University（研究与创新中心（EKIK），奥布达大学）
Ant Colony Optimization (ACO) has been applied in supervised learning in order to induce classification rules as well as decision trees, named Ant-Miners. Although these are competitive classifiers, the stability of these classifiers is an important concern that owes to their stochastic nature. In this paper, to address this issue, an acclaimed machine learning technique named, ensemble of classifiers is applied, where an ACO classifier is used as a base classifier to prepare the ensemble. The main trade-off is, the predictions in the new approach are determined by discovering a group of model…
The lexicase parent selection method selects parents by considering performance on individual data points in random order instead of using a fitness function based on an aggregated data accuracy. While the method has demonstrated promise in genetic programming and more recently in genetic algorithms, its applications in other forms of evolutionary machine learning have not been explored. In this paper, we investigate the use of lexicase parent selection in Learning Classifier Systems (LCS) and study its effect on classification problems in a supervised setting. We further introduce a new varia…
Biomarkers which predict patient's survival can play an important role in medical diagnosis and treatment. How to select the significant biomarkers from hundreds of protein markers is a key step in survival analysis. In this paper a novel method is proposed to detect the prognostic biomarkers of survival in colorectal cancer patients using wavelet analysis, genetic algorithm, and Bayes classifier. One dimensional discrete wavelet transform (DWT) is normally used to reduce the dimensionality of biomedical data. In this study one dimensional continuous wavelet transform (CWT) was proposed to ext…
