Dictionary learning for sparse representations has been successful in many reconstruction tasks. Simplicial learning is an adaptation of dictionary learning, where subspaces become clipped and acquire arbitrary offsets, taking the form of simplices. Such adaptation is achieved through additional constraints on sparse codes. Furthermore, an evolutionary approach can be chosen to determine the number and the dimensionality of simplices composing the simplicial, in which most generative and compact simplicials are favored. This paper proposes an evolutionary simplicial learning method as a generative and compact sparse framework for classification. The proposed approach is first applied on a one-class classification task and it appears as the most reliable method within the considered benchmark. Most surprising results are observed when evolutionary simplicial learning is considered within a multi-class classification task. Since sparse representations are generative in nature, they bear a fundamental problem of not being capable of distinguishing two classes lying on the same subspace. This claim is validated through synthetic experiments and superiority of simplicial learning even as a generative-only approach is demonstrated. Simplicial learning loses its superiority over discriminative methods in high-dimensional cases but can further be modified with discriminative elements to achieve state-of-the-art performance in classification tasks.
We combine two popular optimization approaches to derive learning algorithms for generative models: variational optimization and evolutionary algorithms. The combination is realized for generative models with discrete latents by using truncated posteriors as the family of variational distributions. The variational parameters of truncated posteriors are sets of latent states. By interpreting these states as genomes of individuals and by using the variational lower bound to define a fitness, we can apply evolutionary algorithms to realize the variational loop. The used variational distributions …
University of Technology Sydney（悉尼科技大学）；The University of Sydney（悉尼大学）；Southern University of Science and Technology（南方科技大学）；University of Birmingham（伯明翰大学）
Generative adversarial networks (GAN) have been effective for learning generative models for real-world data. However, existing GANs (GAN and its variants) tend to suffer from training problems such as instability and mode collapse. In this paper, we propose a novel GAN framework called evolutionary generative adversarial networks (E-GAN) for stable GAN training and improved generative performance. Unlike existing GANs, which employ a pre-defined adversarial objective function alternately training a generator and a discriminator, we utilize different adversarial training objectives as mutation…
Generative Adversarial Networks (GAN) is an adversarial model, and it has been demonstrated to be effective for various generative tasks. However, GAN and its variants also suffer from many training problems, such as mode collapse and gradient vanish. In this paper, we firstly propose a general crossover operator, which can be widely applied to GANs using evolutionary strategies. Then we design an evolutionary GAN framework C-GAN based on it. And we combine the crossover operator with evolutionary generative adversarial networks (EGAN) to implement the evolutionary generative adversarial netwo…
Most evolutionary-oriented deep generative models do not explicitly consider the underlying evolutionary dynamics of biological sequences as it is performed within the Bayesian phylogenetic inference framework. In this study, we propose a method for a deep variational Bayesian generative model (EvoVGM) that jointly approximates the true posterior of local evolutionary parameters and generates sequence alignments. Moreover, it is instantiated and tuned for continuous-time Markov chain substitution models such as JC69, K80 and GTR. We train the model via a low-variance stochastic estimator and a…
Generative adversary networks (GANs) suffer from training pathologies such as instability and mode collapse, which mainly arise from a lack of diversity in their adversarial interactions. Co-evolutionary GAN (CoE-GAN) training algorithms have shown to be resilient to these pathologies. This article introduces Mustangs, a spatially distributed CoE-GAN, which fosters diversity by using different loss functions during the training. Experimental analysis on MNIST and CelebA demonstrated that Mustangs trains statistically more accurate generators.
Facebook AI Research（脸书人工智能研究院）；Univ. Littoral Cote d’Opale（利托拉尔滨海大学）；Univ. Konstanz（康斯坦茨大学）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Grenoble INP（格勒诺布尔国立理工学院）；LIG（格勒诺布尔信息学实验室）
We propose to use a quality estimator and evolutionary methods to search the latent space of generative adversarial networks trained on small, difficult datasets, or both. The new method leads to the generation of significantly higher quality images while preserving the original generator's diversity. Human raters preferred an image from the new version with frequency 83.7pc for Cats, 74pc for FashionGen, 70.4pc for Horses, and 69.2pc for Artworks, and minor improvements for the already excellent GANs for faces. This approach applies to any quality scorer and GAN generator.
Southern University of Science and Technology（南方科技大学）；City University of Hong Kong（香港城市大学）；University of Surrey（萨里大学）
Recently, increasing works have proposed to drive evolutionary algorithms using machine learning models. Usually, the performance of such model based evolutionary algorithms is highly dependent on the training qualities of the adopted models. Since it usually requires a certain amount of data (i.e. the candidate solutions generated by the algorithms) for model training, the performance deteriorates rapidly with the increase of the problem scales, due to the curse of dimensionality. To address this issue, we propose a multi-objective evolutionary algorithm driven by the generative adversarial n…
Jilin University（吉林大学）；Northeast Normal University（东北师范大学）
The training of generative adversarial networks (GANs) is usually vulnerable to mode collapse and vanishing gradients. The evolutionary generative adversarial network (E-GAN) attempts to alleviate these issues by optimizing the learning strategy with multiple loss functions. It uses a learning-based evolutionary framework, which develops new mutation operators specifically for general deep neural networks. However, the evaluation mechanism in the fitness function of E-GAN cannot truly reflect the adaptability of individuals to their environment, leading to an inaccurate assessment of the diver…
Recently, more and more works have proposed to drive evolutionary algorithms using machine learning models.Usually, the performance of such model based evolutionary algorithms is highly dependent on the training qualities of the adopted models.Since it usually requires a certain amount of data (i.e. the candidate solutions generated by the algorithms) for model training, the performance deteriorates rapidly with the increase of the problem scales, due to the curse of dimensionality.To address this issue, we propose a multi-objective evolutionary algorithm driven by the generative adversarial n…
Massachusetts Institute of Technology, CSAIL（麻省理工学院计算机科学与人工智能实验室）
Generative Adversarial Networks (GANs) are popular tools for generative modeling. The dynamics of their adversarial learning give rise to convergence pathologies during training such as mode and discriminator collapse. In machine learning, ensembles of predictors demonstrate better results than a single predictor for many tasks. In this study, we apply two evolutionary algorithms (EAs) to create ensembles to re-purpose generative models, i.e., given a set of heterogeneous generators that were optimized for one objective (e.g., minimize Frechet Inception Distance), create ensembles of them for …
Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research (A*STAR), Singapore（前沿人工智能研究中心（CFAR），科技研究局（A*STAR），新加坡）；School of Mechanical Sciences, Indian Institute of Technology (IIT) Goa（机械科学学院，印度理工学院（IIT） Goa）；School of Computer Science and Technology, Dalian University of Technology (DLUT)（计算机科学与技术学院，大连理工大学）；College of Computing and Data Science (CCDS), Nanyang Technological University (NTU)（计算与数据科学学院（CCDS），南洋理工大学）；Honda Research Institute Europe (HRI-EU), Offenbach am Main, Germany（本田欧洲研究院（HRI-EU），德国Offenbach am Main）；Data Science and Artificial Intelligence Research Centre, School of Computer Science and Engineering, Nanyang Technological University (NTU)（数据科学与人工智能研究中心，计算机科学与工程学院，南洋理工大学）
Generating multiple categories of texts is a challenging task and draws more and more attention. Since generative adversarial nets (GANs) have shown competitive results on general text generation, they are extended for category text generation in some previous works. However, the complicated model structures and learning strategies limit their performance and exacerbate the training instability. This paper proposes a category-aware GAN (CatGAN) which consists of an efficient category-aware model for category text generation and a hierarchical evolutionary learning algorithm for training our mo…
LLM2TEA: An Agentic AI Designer for Discovery with Generative Evolutionary Multitasking
1 College of Computing \& Data Science (CCDS), Nanyang Technological University (NTU), Singapore 2 Honda Research Institute Europe (HRI-EU), Offenbach am Main, Germany 3 Centre for Frontier AI Research (CFAR), Agency for Science, Technology
Allen Discovery Center at Tufts University（塔夫茨大学艾伦发现中心）；Institute for Theoretical Physics, TU Wien（维也纳技术大学理论物理研究所）；Wyss Institute for Biologically Inspired Engineering at Harvard University（哈佛大学威斯生物启发工程研究所）
Flow-Based Generative Emulation of Grids of Stellar Evolutionary Models
Kavli Institute for Astrophysics and Space Research, Massachusetts Institute of Technology（麻省理工学院卡弗里天体物理与空间研究所）；Institute for Astronomy, University of Hawai‘i（夏威夷大学天文研究所）
University of Bucharest（布加勒斯特大学）；University Politehnica of Bucharest（布加勒斯特理工大学）
We study the task of replicating the functionality of black-box neural models, for which we only know the output class probabilities provided for a set of input images. We assume back-propagation through the black-box model is not possible and its training images are not available, e.g. the model could be exposed only through an API. In this context, we present a teacher-student framework that can distill the black-box (teacher) model into a student model with minimal accuracy loss. To generate useful data samples for training the student, our framework (i) learns to generate images on a proxy…
Tool-Aided Evolutionary LLM for Generative Policy Toward Efficient Resource Management in Wireless Federated Learning
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；Zhejiang Lab（浙江实验室）；University of Manitoba（曼尼托巴大学）；Macau University of Science and Technology（澳门科学理工学院）
Courant Institute of Mathematical Sciences, NYU（纽约大学库朗数学科学研究所）
We present two main contributions which help us in leveraging the theory of graphons for modeling evolutionary processes. We show a generative model for digraphons using a finite basis of subgraphs, which is representative of biological networks with evolution by duplication. We show a simple MAP estimate on the Bayesian non parametric model using the Dirichlet Chinese restaurant process representation, with the help of a Gibbs sampling algorithm to infer the prior. Next we show an efficient implementation to do simulations on finite basis segmentations of digraphons. This implementation is us…
Massachusetts Institute of Technology（麻省理工学院）；University of Malaga（马拉加大学）
Generative adversarial networks (GANs) exhibit training pathologies that can lead to convergence-related degenerative behaviors, whereas spatially-distributed, coevolutionary algorithms (CEAs) for GAN training, e.g. Lipizzaner, are empirically robust to them. The robustness arises from diversity that occurs by training populations of generators and discriminators in each cell of a toroidal grid. Communication, where signals in the form of parameters of the best GAN in a cell propagate in four directions: North, South, West, and East, also plays a role, by communicating adaptations that are bot…
LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search
University of Oslo（奥斯陆大学）；Institute for Clarity in Documentation（文献清晰度研究所）；Inria Paris-Rocquencourt（法国国家信息与自动化研究所巴黎-罗康分校）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕尔默研究实验室）
Hierarchical Conditioning of Diffusion Models Using Tree-of-Life for Studying Species Evolution
Virginia Tech（弗吉尼亚理工大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；University of California, Irvine（加利福尼亚大学欧文分校）；Tulane University（杜兰大学）；Battelle（巴特尔公司）；Duke University（杜克大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；The Ohio State University（俄亥俄州立大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
