首页/论文/2001.10178

arXiv 2001.101782020-01-28cs.NEcs.AIcs.LG

An Adaptive and Near Parameter-free Evolutionary Computation Approach Towards True Automation in AutoML

Victoria University of Wellington（惠灵顿维多利亚大学）

Benjamin Patrick Evans, Bing Xue, Mengjie Zhang

备注：: 18 pages (single column), 2 figures
DOI：: 10.1109/CEC48606.2020.9185770

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

A common claim of evolutionary computation methods is that they can achieve good results without the need for human intervention. However, one criticism of this is that there are still hyperparameters which must be tuned in order to achieve good performance. In this work, we propose a near "parameter-free" genetic programming approach, which adapts the hyperparameter values throughout evolution without ever needing to be specified manually. We apply this to the area of automated machine learning (by extending TPOT), to produce pipelines which can effectively be claimed to be free from human input, and show that the results are competitive with existing state-of-the-art which use hand-selected hyperparameter values. Pipelines begin with a randomly chosen estimator and evolve to competitive pipelines automatically. This work moves towards a truly automatic approach to AutoML.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Improvement of Computational Performance of Evolutionary AutoML in a Heterogeneous Environment

ITMO University（圣彼得堡国立信息技术机械与光学大学）

Resource-intensive computations are a major factor that limits the effectiveness of automated machine learning solutions. In the paper, we propose a modular approach that can be used to increase the quality of evolutionary optimization for modelling pipelines with a graph-based structure. It consists of several stages - parallelization, caching and evaluation. Heterogeneous and remote resources can be involved in the evaluation stage. The conducted experiments confirm the correctness and effectiveness of the proposed approach. The implemented algorithms are available as a part of the open-sour…

2023-01-12cs.LGcs.NEcs.PF

Identifying and Harnessing the Building Blocks of Machine Learning Pipelines for Sensible Initialization of a Data Science Automation Tool

Institute for Biomedical Informatics, University of Pennsylvania（宾夕法尼亚大学生物医学信息学研究所）

As data science continues to grow in popularity, there will be an increasing need to make data science tools more scalable, flexible, and accessible. In particular, automated machine learning (AutoML) systems seek to automate the process of designing and optimizing machine learning pipelines. In this chapter, we present a genetic programming-based AutoML system called TPOT that optimizes a series of feature preprocessors and machine learning models with the goal of maximizing classification accuracy on a supervised classification problem. Further, we analyze a large database of pipelines that …

2016-07-29cs.AIcs.LGcs.NE

Evolutionary Neural AutoML for Deep Learning

Deep neural networks (DNNs) have produced state-of-the-art results in many benchmarks and problem domains. However, the success of DNNs depends on the proper configuration of its architecture and hyperparameters. Such a configuration is difficult and as a result, DNNs are often not used to their full potential. In addition, DNNs in commercial applications often need to satisfy real-world design constraints such as size or number of parameters. To make configuration easier, automatic machine learning (AutoML) systems for deep learning have been developed, focusing mostly on optimization of hype…

2019-04-09cs.NE

Autostacker: A Compositional Evolutionary Learning System

Columbia University（哥伦比亚大学）；University of Chicago（芝加哥大学）

We introduce an automatic machine learning (AutoML) modeling architecture called Autostacker, which combines an innovative hierarchical stacking architecture and an Evolutionary Algorithm (EA) to perform efficient parameter search. Neither prior domain knowledge about the data nor feature preprocessing is needed. Using EA, Autostacker quickly evolves candidate pipelines with high predictive accuracy. These pipelines can be used as is or as a starting point for human experts to build on. Autostacker finds innovative combinations and structures of machine learning models, rather than selecting a…

2018-03-02cs.LGcs.NEstat.ML

集成进化自动化机器学习与结构敏感性分析的复合管道

Integration Of Evolutionary Automated Machine Learning With Structural Sensitivity Analysis For Composite Pipelines

ITMO University（圣彼得堡国立信息技术、机械学与光学研究大学）；Huawei Technologies Co Ltd（华为技术有限公司）

提出EVOSA方法，结合敏感性分析与进化AutoML，优化灵活管道结构，提高鲁棒性和可解释性，并在多领域任务中验证有效性。

2023-12-22cs.LGcs.NE

Evolutionary Architecture Search for Graph Neural Networks

Automated machine learning (AutoML) has seen a resurgence in interest with the boom of deep learning over the past decade. In particular, Neural Architecture Search (NAS) has seen significant attention throughout the AutoML research community, and has pushed forward the state-of-the-art in a number of neural models to address grid-like data such as texts and images. However, very litter work has been done about Graph Neural Networks (GNN) learning on unstructured network data. Given the huge number of choices and combinations of components such as aggregator and activation function, determinin…

2020-09-21cs.NE

更多相关论文收起更多论文

用语言模型演化用于自动机器学习的可执行流水线程序

Evolving Executable Pipeline Programs for AutoML with Language Models

LACE是首个以代码形式构建通用表格型AutoML的框架，用语言模型演化可执行流水线，在68个OpenML分类任务上表现优异，其核心贡献是提供可直接复用的代码级搜索空间。

2026-08-18cs.LGcs.NE

Multi-Objective Evolutionary Design of Composite Data-Driven Models

Natural Systems Simulation Lab ITMO University（ITMO大学自然系统模拟实验室）

In this paper, a multi-objective approach for the design of composite data-driven mathematical models is proposed. It allows automating the identification of graph-based heterogeneous pipelines that consist of different blocks: machine learning models, data preprocessing blocks, etc. The implemented approach is based on a parameter-free genetic algorithm (GA) for model design called GPComp@Free. It is developed to be part of automated machine learning solutions and to increase the efficiency of the modeling pipeline automation. A set of experiments was conducted to verify the correctness and e…

2021-05-17cs.AIcs.LGcs.NE

Unified Functional Hashing in Automatic Machine Learning

Google Research（谷歌研究院）；Google（谷歌）

The field of Automatic Machine Learning (AutoML) has recently attained impressive results, including the discovery of state-of-the-art machine learning solutions, such as neural image classifiers. This is often done by applying an evolutionary search method, which samples multiple candidate solutions from a large space and evaluates the quality of each candidate through a long training process. As a result, the search tends to be slow. In this paper, we show that large efficiency gains can be obtained by employing a fast unified functional hash, especially through the functional equivalence ca…

2023-02-10cs.LGcs.NE

DarwinML: A Graph-based Evolutionary Algorithm for Automated Machine Learning

Xidian University（西安电子科技大学）；Intelligence Qubic (Beijing) Technology Co., Ltd（智立方（北京）科技有限公司）

As an emerging field, Automated Machine Learning (AutoML) aims to reduce or eliminate manual operations that require expertise in machine learning. In this paper, a graph-based architecture is employed to represent flexible combinations of ML models, which provides a large searching space compared to tree-based and stacking-based architectures. Based on this, an evolutionary algorithm is proposed to search for the best architecture, where the mutation and heredity operators are the key for architecture evolution. With Bayesian hyper-parameter optimization, the proposed approach can automate th…

2018-11-20cs.LGcs.NEstat.ML

LLM引导的进化——模型推进模型的自动化

LLM Guided Evolution -- The Automation of Models Advancing Models

Georgia Tech Research Institute（佐治亚理工学院研究院）

本文提出利用大语言模型直接修改代码的引导进化框架，通过思维进化技术实现自我反馈，在保持模型紧凑性的同时将ExquisiteNetV2准确率从92.52%提升至93.34%，展示了模型自主进化的潜力。

2024-03-18cs.LGcs.NE

EvoTorch：Python中可扩展的进化计算

EvoTorch: Scalable Evolutionary Computation in Python

NNAISENSE

针对现代高维优化问题对可扩展进化算法实现的需求，本文提出基于PyTorch的进化计算库EvoTorch，支持GPU加速与高度并行化，允许用户使用熟悉的API定义优化问题。

2023-05-21cs.AIcs.NE

Layered TPOT: Speeding up Tree-based Pipeline Optimization

Technische Universiteit Eindhoven（埃因霍温理工大学）；University of Pennsylvania（宾夕法尼亚大学）

With the demand for machine learning increasing, so does the demand for tools which make it easier to use. Automated machine learning (AutoML) tools have been developed to address this need, such as the Tree-Based Pipeline Optimization Tool (TPOT) which uses genetic programming to build optimal pipelines. We introduce Layered TPOT, a modification to TPOT which aims to create pipelines equally good as the original, but in significantly less time. This approach evaluates candidate pipelines on increasingly large subsets of the data according to their fitness, using a modified evolutionary algori…

2018-03-12cs.AIcs.NE

Evolutionary Algorithms in Approximate Computing: A Survey

In recent years, many design automation methods have been developed to routinely create approximate implementations of circuits and programs that show excellent trade-offs between the quality of output and required resources. This paper deals with evolutionary approximation as one of the popular approximation methods. The paper provides the first survey of evolutionary algorithm (EA)-based approaches applied in the context of approximate computing. The survey reveals that EAs are primarily applied as multi-objective optimizers. We propose to divide these approaches into two main classes: (i) p…

2021-08-16cs.NE

评估进化计算中的可重复性：通过人类和LLM基于的评估进行案例研究

Assessing Reproducibility in Evolutionary Computation: A Case Study using Human- and LLM-based Assessment

DPIA, University of Udine（DPIA，乌迪内大学）；LIACS, Leiden University（LIACS，莱顿大学）

本文通过人类和LLM评估，研究了进化计算领域论文的可重复性现状，提出RECAP系统并发现平均完整性得分较低，自动化评估与人工评估一致度较高。

2026-02-05cs.AIcs.NEcs.SE

A Survey on Neural Architecture Search

IBM Research AI（IBM研究院人工智能部门）；IBM T.J. Watson Research Center（IBM托马斯·J·沃森研究中心）

The growing interest in both the automation of machine learning and deep learning has inevitably led to the development of a wide variety of automated methods for neural architecture search. The choice of the network architecture has proven to be critical, and many advances in deep learning spring from its immediate improvements. However, deep learning techniques are computationally intensive and their application requires a high level of domain knowledge. Therefore, even partial automation of this process helps to make deep learning more accessible to both researchers and practitioners. With …

2019-06-18cs.CVcs.LGcs.NE