Neural networks are sensitive to hyper-parameter and architecture choices. Automated Machine Learning (AutoML) is a promising paradigm for automating these choices. Current ML software libraries, however, are quite limited in handling the dynamic interactions among the components of AutoML. For example, efficientNAS algorithms, such as ENAS and DARTS, typically require an implementation coupling between the search space and search algorithm, the two key components in AutoML. Furthermore, implementing a complex search flow, such as searching architectures within a loop of searching hardware configurations, is difficult. To summarize, changing the search space, search algorithm, or search flow in current ML libraries usually requires a significant change in the program logic. In this paper, we introduce a new way of programming AutoML based on symbolic programming. Under this paradigm, ML programs are mutable, thus can be manipulated easily by another program. As a result, AutoML can be reformulated as an automated process of symbolic manipulation. With this formulation, we decouple the triangle of the search algorithm, the search space and the child program. This decoupling makes it easy to change the search space and search algorithm (without and with weight sharing), as well as to add search capabilities to existing code and implement complex search flows. We then introduce PyGlove, a new Python library that implements this paradigm. Through case studies on ImageNet and NAS-Bench-101, we show that with PyGlove users can easily convert a static program into a search space, quickly iterate on the search spaces and search algorithms, and craft complex search flows to achieve better results.
Hash-Based Tree Similarity and Simplification in Genetic Programming for Symbolic Regression
Josef Ressel Centre for Symbolic Regression（约瑟夫·雷塞尔符号回归中心）；University of Applied Sciences Upper Austria（上奥地利应用科学大学）；Johannes Kepler University（约翰内斯·开普勒大学）
Discovering Continuous-Time Memory-Based Symbolic Policies using Genetic Programming
Department of Machine Learning and Neural Computing（机器学习与神经计算系）；Donders Institute for Brain, Cognition and Behaviour（大脑、认知与行为研究所）；Radboud University（拉德堡德大学）
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming
Carnegie Mellon University（卡内基梅隆大学）；University of Pennsylvania（宾夕法尼亚大学）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Petuum Inc.（Petuum公司）
Evaluation of Population Initialization Methods for Genetic Programming-based Symbolic Regression
Heuristic and Evolutionary Algorithms Laboratory, University of Applied Sciences Upper Austria（上奥地利应用科学大学启发式与进化算法实验室）；Astrophysics, University of Oxford（牛津大学天体物理学系）；Institute of Cosmology & Gravitation, University of Portsmouth（朴茨茅斯大学宇宙学与引力研究所）
ANDRE: An Attention-based Neuro-symbolic Differentiable Rule Extractor for Inductive Logic Programming
Dept. of Mechanical and Aerospace Engineering, George Washington University, USA（机械与航空航天工程系，乔治华盛顿大学）；Dept. of Mechanical Engineering Sciences, University of Surrey, UK（机械工程科学系，萨里大学）
Symbolic Graphics Programming with Large Language Models
The Chinese University of Hong Kong（香港中文大学）；Westlake University（西湖大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Data Aggregation for Reducing Training Data in Symbolic Regression
Josef Ressel Center for Symbolic Regression（约瑟夫·雷塞尔符号回归中心）；Heuristic and Evolutionary Algorithms Laboratory（启发式与进化算法实验室）；University of Applied Sciences Upper Austria（上奥地利应用科学大学）
Efficient Constant Optimization for Symbolic Regression with GPU-Accelerated Tree-Based Genetic Programming
The Hong Kong Polytechnic University（香港理工大学）；University of Washington（华盛顿大学）；Amazon Web Services（亚马逊网络服务）；Jiangxi University of Finance and Economics（江西财经大学）；EEO Education Technology（EEO教育科技）；Chongqing Medical University（重庆医科大学）；Eastern Institute of Technology（东方理工学院）
QiMeng-Xpiler: Transcompiling Tensor Programs for Deep Learning Systems with a Neural-Symbolic Approach
University of Science and Technology of China（科学技术大学）；Cambricon Technologies（Cambricon科技）；SKL of Processors, Institute of Computing Technology, Chinese Academy of Sciences（处理器实验室，计算技术研究所，中国科学院）；Institute of Software, Chinese Academy of Sciences（软件研究所，中国科学院）
Tecnológico Nacional de México/IT de Tijuana（墨西哥国家理工学院/蒂胡ana信息技术学院）；LASIGE, Department of Informatics, Faculty of Sciences, University of Lisbon（里斯本大学科学学院信息系LASIGE）
