We develop a general approach to distill symbolic representations of a learned deep model by introducing strong inductive biases. We focus on Graph Neural Networks (GNNs). The technique works as follows: we first encourage sparse latent representations when we train a GNN in a supervised setting, then we apply symbolic regression to components of the learned model to extract explicit physical relations. We find the correct known equations, including force laws and Hamiltonians, can be extracted from the neural network. We then apply our method to a non-trivial cosmology example-a detailed dark matter simulation-and discover a new analytic formula which can predict the concentration of dark matter from the mass distribution of nearby cosmic structures. The symbolic expressions extracted from the GNN using our technique also generalized to out-of-distribution data better than the GNN itself. Our approach offers alternative directions for interpreting neural networks and discovering novel physical principles from the representations they learn.
Discovering equations from data: symbolic regression in dynamical systems
Institute of Sciences Mathematics and Computation, Universidade de São Paulo-Campus de São Carlos（数学与计算科学研究院，圣保罗大学-萨o卡洛斯校区）；Statistic Department, State University of Maringá（统计系，马林加州立大学）；Department of Mathematics Applied and Statistics, Institute of Sciences Mathematics and Computation, Universidade de São Paulo-Campus de São Carlos（应用数学与统计系，数学与计算科学研究院，圣保罗大学-萨o卡洛斯校区）
Discovering Continuous-Time Memory-Based Symbolic Policies using Genetic Programming
Department of Machine Learning and Neural Computing（机器学习与神经计算系）；Donders Institute for Brain, Cognition and Behaviour（大脑、认知与行为研究所）；Radboud University（拉德堡德大学）
Neuro-Symbolic Imitation Learning: Discovering Symbolic Abstractions for Skill Learning
Intelligent Autonomous Systems, TU Darmstadt, Germany（图腾达姆施塔特大学智能自主系统研究所）；German Research Center for AI, Germany（德国人工智能研究中心）；Hessian Centre for Artificial Intelligence, Germany（黑森州人工智能中心）；Honda Research Institute EU, Germany（本田欧洲研究院）
Discovering physical laws with parallel symbolic enumeration
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）；State Key Laboratory of Nonlinear Mechanics, Institute of Mechanics, Chinese Academy of Sciences（中国科学院力学研究所非线性力学重点实验室）；Department of Computer Science and Engineering, HKUST（香港科技大学计算机科学与工程系）
ANDRE: An Attention-based Neuro-symbolic Differentiable Rule Extractor for Inductive Logic Programming
Dept. of Mechanical and Aerospace Engineering, George Washington University, USA（机械与航空航天工程系，乔治华盛顿大学）；Dept. of Mechanical Engineering Sciences, University of Surrey, UK（机械工程科学系，萨里大学）
The Road to Learning Explainable Inverse Kinematic Models: Graph Neural Networks as Inductive Bias for Symbolic Regression
Faculty of Computer Science, Otto-von-Guericke-University Magdeburg（马格德堡奥托·冯·格里克大学计算机科学学院）；Fraunhofer Institute for Transportation and Infrastructure Systems IVI（弗劳恩霍夫运输与基础设施系统研究所IVI）
Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)
Peng Cheng Laboratory（鹏城实验室）；University of Washington（华盛顿大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）
Discovering an interpretable mathematical expression for a full wind-turbine wake with artificial intelligence enhanced symbolic regression
School of Ocean and Civil Engineering, Shanghai Jiao Tong University（上海交通大学海洋与土木工程学院）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）
Physics Symbolic Learner for Discovering Ground-Motion Models Via NGA-West2 Database
Beijing University of Technology（北京工业大学）；Institute of Geophysics, China Earthquake Administration（中国地震局地球物理研究所）；Institute of Geomechanics, Chinese Academy of Geological Sciences（中国地质科学院地质力学研究所）
Weak-PDE-Net: Discovering Open-Form PDEs via Differentiable Symbolic Networks and Weak Formulation
School of Mathematical Sciences, East China Normal University（东华大学数学科学学院）；Institute of Applied Physics and Computational Mathematics（应用物理与计算数学研究所）；Institute of Artificial Intelligence, Beihang University（北航人工智能研究所）；Shanghai Zhangjiang Institute of Mathematics（上海张江数学研究所）；Academy for Advanced Interdisciplinary Studies, Northeast Normal University（东北师范大学先进交叉学科研究院）
Discovering Ordinary Differential Equations with LLM-Based Qualitative and Quantitative Evaluation
Department of Artificial Intelligence, Chung-Ang University, Seoul, Republic of Korea（韩国首尔 Chung-Ang 大学人工智能系）；Daejin University, Pocheon, Republic of Korea（韩国坡州市 Daejin 大学）
Scalable Neural Symbolic Regression using Control Variables
School of Computer Science（计算机科学学院）；GaTech（佐治亚理工学院）；School of Computing and Data Science（计算与数据科学学院）；UIUC（伊利诺伊大学厄巴纳-香槟分校）；William & Mary（威廉玛丽学院）；Department of EECS（电子工程与计算机科学系）；University of Tennessee, Knoxville（田纳西大学诺克斯维尔分校）；Wake Forest University（维克森林大学）
Discovering Mathematical Formulas from Data via GPT-guided Monte Carlo Tree Search
ANNLAB, Institute of Semiconductors, Chinese Academy of Sciences（中国科学院半导体研究所ANNLAB）；School of Integrated Circuits, University of Chinese Academy of Sciences（中国科学院大学集成电路学院）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）
