Feature engineering, a crucial step of machine learning, aims to extract useful features from raw data to improve data quality. In recent years, great efforts have been devoted to Automated Feature Engineering (AutoFE) to replace expensive human labor. However, existing methods are computationally demanding due to treating AutoFE as a coarse-grained black-box optimization problem over a discrete space. In this work, we propose an efficient gradient-based method called DIFER to perform differentiable automated feature engineering in a continuous vector space. DIFER selects potential features based on evolutionary algorithm and leverages an encoder-predictor-decoder controller to optimize existing features. We map features into the continuous vector space via the encoder, optimize the embedding along the gradient direction induced by the predicted score, and recover better features from the optimized embedding by the decoder. Extensive experiments on classification and regression datasets demonstrate that DIFER can significantly improve the performance of various machine learning algorithms and outperform current state-of-the-art AutoFE methods in terms of both efficiency and performance.
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；Anyvision（Anyvision公司）；Queens University of Belfast（贝尔法斯特女王大学）；School of Informatics, The University of Edinburgh（爱丁堡大学信息学院）
Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation
Wells Fargo（摩根大通）；Graduate Institute of Applied Physics（应用物理研究所）；Imperial College London（帝国理工学院）；Centre for Quantum Engineering, Science and Technology (QuEST)（量子工程、科学与技术中心）；Hon Hai (Foxconn) Research Institute（鸿海（富士康）研究院）；Quantum Information Center（量子信息中心）；Master Program in Intelligent Computing and Big Data（智能计算与大数据硕士项目）；Jij inc.（Jij公司）
FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking
Beijing Normal University（北京师范大学）；Southern University of Science and Technology（南方科技大学）；University of Illinois Chicago（伊利诺伊大学香槟分校）；Politecnico di Milano（米兰理工大学）；Universita’ Della Svizzera Italiana（瑞士意大利大学）
Automatic feature selection and weighting in molecular systems using Differentiable Information Imbalance
International School for Advanced Studies (SISSA)（国际高等研究院（SISSA））；The Institute of Science and Technology Austria (ISTA)（奥地利科学技术研究所）；University of Cal（卡尔大学）
DiffRenderGAN: Addressing Training Data Scarcity in Deep Segmentation Networks for Quantitative Nanomaterial Analysis through Differentiable Rendering and Generative Modelling
Department Artificial Intelligence in Biomedical Engineering（人工智能生物医学工程部门）；Friedrich-Alexander-University Erlangen-Nürnberg（埃朗根-纽伦堡弗里德里希-亚历山大大学）；Pattern Recognition Lab（模式识别实验室）；Institute for Nanotechnology and Correlative Microscopy（纳米技术与相关显微镜研究所）；Lucid Concepts AG（Lucid Concepts公司）；Fraunhofer Institute for Ceramic Technologies and Systems（弗劳恩霍夫陶瓷技术与系统研究所）；Freie Universität Berlin（柏林自由大学）；Max Planck Institute for the Science of Light（马克斯·普朗克光科学研究所）；Julius-Maximilians-University Würzburg（乌尔姆-马克斯·普朗克大学）
