Sparse regression on a library of candidate features has developed as the prime method to discover the partial differential equation underlying a spatio-temporal data-set. These features consist of higher order derivatives, limiting model discovery to densely sampled data-sets with low noise. Neural network-based approaches circumvent this limit by constructing a surrogate model of the data, but have to date ignored advances in sparse regression algorithms. In this paper we present a modular framework that dynamically determines the sparsity pattern of a deep-learning based surrogate using any sparse regression technique. Using our new approach, we introduce a new constraint on the neural network and show how a different network architecture and sparsity estimator improve model discovery accuracy and convergence on several benchmark examples. Our framework is available at \url{https://github.com/PhIMaL/DeePyMoD}
In this paper, we consider the problem of learning prediction models for spatiotemporal physical processes driven by unknown partial differential equations (PDEs). We propose a deep learning framework that learns the underlying dynamics and predicts its evolution using sparsely distributed data sites. Deep learning has shown promising results in modeling physical dynamics in recent years. However, most of the existing deep learning methods for modeling physical dynamics either focus on solving known PDEs or require data in a dense grid when the governing PDEs are unknown. In contrast, our meth…
Unrolled-SINDy: A Stable Explicit Method for Non linear PDE Discovery from Sparsely Sampled Data
1 Jean Monnet University Saint-\'Etienne, CNRS, Institut d'Optique Graduate School, Inria, Hubert Curien Laboratory UMR 5516, F-42023, SAINT-\'ETIENNE, France
Université de Paris（巴黎大学）；INSERM U1284（法国国家健康与医学研究院 U1284）；CRI（计算研究中心）；IBM Research Paris-Saclay（IBM 巴黎-萨克雷研究院）
Automated model discovery of partial differential equations (PDEs) usually considers a single experiment or dataset to infer the underlying governing equations. In practice, experiments have inherent natural variability in parameters, initial and boundary conditions that cannot be simply averaged out. We introduce a randomised adaptive group Lasso sparsity estimator to promote grouped sparsity and implement it in a deep learning based PDE discovery framework. It allows to create a learning bias that implies the a priori assumption that all experiments can be explained by the same underlying PD…
College of Engineering, Peking University（北京大学工学院）；School of Environmental Science and Engineering, Southern University of Science and Technology（南方科技大学环境科学与工程学院）；Peng Cheng Laboratory（鹏城实验室）
Data-driven methods have recently been developed to discover underlying partial differential equations (PDEs) of physical problems. However, for these methods, a complete candidate library of potential terms in a PDE are usually required. To overcome this limitation, we propose a novel framework combining deep learning and genetic algorithm, called DLGA-PDE, for discovering PDEs. In the proposed framework, a deep neural network that is trained with available data of a physical problem is utilized to generate meta-data and calculate derivatives, and the genetic algorithm is then employed to dis…
In this paper, we address the problem of predicting complex, nonlinear spatiotemporal dynamics when available data is recorded at irregularly-spaced sparse spatial locations. Most of the existing deep learning models for modeling spatiotemporal dynamics are either designed for data in a regular grid or struggle to uncover the spatial relations from sparse and irregularly-spaced data sites. We propose a deep learning model that learns to predict unknown spatiotemporal dynamics using data from sparsely-distributed data sites. We base our approach on Radial Basis Function (RBF) collocation method…
Generative Discovery of Partial Differential Equations by Learning from Math Handbooks
Eastern Institute of Technology（东方理工）；Tsinghua University（清华大学）；Ocean University of China（中国海洋大学）；Imperial College London（伦敦帝国理工学院）；University of Oxford（牛津大学）；University of Manchester（曼彻斯特大学）；Peking University（北京大学）；Lingnan University（岭南大学）
LLM-PDESR: Robust PDE Discovery via Subdomain Weighted Residuals and LLM-Guided Symbolic Hypothesis Generation
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；School of Big Data and Artificial Intelligence, Guangdong University of Finance and Economics（广东财经大学大数据与人工智能学院）；Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University（吉林大学符号计算与知识工程教育部重点实验室）；School of Computer Science, Zhuhai College of Science and Technology（珠海科技学院计算机科学学院）；Department of Mechanical Engineering, National University of Singapore（新加坡国立大学机械工程系）
Discovery of partial differential equations from highly noisy and sparse data with physics-informed information criterion
College of Engineering, Peking University（北京大学工学院）；Institute of Applied Physics and Computational Mathematics（应用物理与计算数学研究所）；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）
Mechanistic PDE Networks for Discovery of Governing Equations
Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands（阿姆斯特丹大学信息技术研究所）；Institute of Science and Technology, Klosterneuburg, Austria（科技研究院）
Huazhong University of Science and Technology（华中科技大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Georgia Institute of Technology（佐治亚理工学院）；Wuhan University（武汉大学）；University of Cambridge（剑桥大学）；University of Luxembourg（卢森堡大学）；Weill Cornell Medical College（威尔康奈尔医学院）
The study presents a general framework for discovering underlying Partial Differential Equations (PDEs) using measured spatiotemporal data. The method, called Sparse Spatiotemporal System Discovery ($\text{S}^3\text{d}$), decides which physical terms are necessary and which can be removed (because they are physically negligible in the sense that they do not affect the dynamics too much) from a pool of candidate functions. The method is built on the recent development of Sparse Bayesian Learning; which enforces the sparsity in the to-be-identified PDEs, and therefore can balance the model compl…
Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）
Data-driven discovery of partial differential equations (PDEs) has attracted increasing attention in recent years. Although significant progress has been made, certain unresolved issues remain. For example, for PDEs with high-order derivatives, the performance of existing methods is unsatisfactory, especially when the data are sparse and noisy. It is also difficult to discover heterogeneous parametric PDEs where heterogeneous parameters are embedded in the partial differential operators. In this work, a new framework combining deep-learning and integral form is proposed to handle the above-men…
Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations
Carnegie Mellon University（卡内基梅隆大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）；Hefei University of Technology（合肥工业大学）；RealAI（瑞莱智慧）；Tsinghua University（清华大学）
Peking University（北京大学）；Eastern Institute for Advanced Study（东方高等研究院）；Eastern Institute of Technology（东方理工大学）；National Center for Applied Mathematics Shenzhen（深圳国家应用数学中心）；Southern University of Science and Technology（南方科技大学）
The working mechanisms of complex natural systems tend to abide by concise and profound partial differential equations (PDEs). Methods that directly mine equations from data are called PDE discovery, which reveals consistent physical laws and facilitates our adaptive interaction with the natural world. In this paper, an enhanced deep reinforcement-learning framework is proposed to uncover symbolically concise open-form PDEs with little prior knowledge. Particularly, based on a symbol library of basic operators and operands, a structure-aware recurrent neural network agent is designed and seaml…
Qu & Co B.V.（Qu & Co 公司）；Czech Technical University（捷克技术大学）；University of Exeter（埃克塞特大学）
Quantum computing promises to speed up some of the most challenging problems in science and engineering. Quantum algorithms have been proposed showing theoretical advantages in applications ranging from chemistry to logistics optimization. Many problems appearing in science and engineering can be rewritten as a set of differential equations. Quantum algorithms for solving differential equations have shown a provable advantage in the fault-tolerant quantum computing regime, where deep and wide quantum circuits can be used to solve large linear systems like partial differential equations (PDEs) …
Weak-PDE-Net: Discovering Open-Form PDEs via Differentiable Symbolic Networks and Weak Formulation
School of Mathematical Sciences, East China Normal University（东华大学数学科学学院）；Institute of Applied Physics and Computational Mathematics（应用物理与计算数学研究所）；Institute of Artificial Intelligence, Beihang University（北航人工智能研究所）；Shanghai Zhangjiang Institute of Mathematics（上海张江数学研究所）；Academy for Advanced Interdisciplinary Studies, Northeast Normal University（东北师范大学先进交叉学科研究院）
Adaptation of uncertainty-penalized Bayesian information criterion for parametric partial differential equation discovery
Graduate School of Information Science and Technology, Osaka University（大阪大学信息科学与技术研究生院）；SANKEN (The Institute of Scientific and Industrial Research), Osaka University（大阪大学产业科学研究所（SANKEN））
Max Planck Institute for Dynamics of Complex Technical Systems（马克斯·普朗克复杂技术系统动力学研究所）
Discovering dynamical models to describe underlying dynamical behavior is essential to draw decisive conclusions and engineering studies, e.g., optimizing a process. Experimental data availability notwithstanding has increased significantly, but interpretable and explainable models in science and engineering yet remain incomprehensible. In this work, we blend machine learning and dictionary-based learning with numerical analysis tools to discover governing differential equations from noisy and sparsely-sampled measurement data. We utilize the fact that given a dictionary containing huge candid…
Equations governing physico-chemical processes are usually known at microscopic spatial scales, yet one suspects that there exist equations, e.g. in the form of Partial Differential Equations (PDEs), that can explain the system evolution at much coarser, meso- or macroscopic length scales. Discovering those coarse-grained effective PDEs can lead to considerable savings in computation-intensive tasks like prediction or control. We propose a framework combining artificial neural networks with multiscale computation, in the form of equation-free numerics, for efficient discovery of such macro-sca…
Yale Child Study Center, School of Medicine, Yale University（耶鲁大学医学院耶鲁儿童研究中心）；Paul G. Allen School of Computer Science and Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Seattle Children’s Research Institute（西雅图儿童研究所）；School of Medicine, University of Washington（华盛顿大学医学院）
In genomic analysis, biomarker discovery, image recognition, and other systems involving machine learning, input variables can often be organized into different groups by their source or semantic category. Eliminating some groups of variables can expedite the process of data acquisition and avoid over-fitting. Researchers have used the group lasso to ensure group sparsity in linear models and have extended it to create compact neural networks in meta-learning. Different from previous studies, we use multi-layer non-linear neural networks to find sparse groups for input variables. We propose a …
Peng Cheng Laboratory（鹏城实验室）；University of Washington（华盛顿大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）
Partial differential equations (PDEs) are concise and understandable representations of domain knowledge, which are essential for deepening our understanding of physical processes and predicting future responses. However, the PDEs of many real-world problems are uncertain, which calls for PDE discovery. We propose the symbolic genetic algorithm (SGA-PDE) to discover open-form PDEs directly from data without prior knowledge about the equation structure. SGA-PDE focuses on the representation and optimization of PDE. Firstly, SGA-PDE uses symbolic mathematics to realize the flexible representatio…
Center for Research and Interdisciplinarity (CRI)（跨学科研究中心（CRI））；Université de Paris, INSERM U1284（巴黎大学，法国国家健康与医学研究院U1284研究所）；F-75006 Paris, France（法国巴黎75006）
Discovering the partial differential equations underlying spatio-temporal datasets from very limited and highly noisy observations is of paramount interest in many scientific fields. However, it remains an open question to know when model discovery algorithms based on sparse regression can actually recover the underlying physical processes. In this work, we show the design matrices used to infer the equations by sparse regression can violate the irrepresentability condition (IRC) of the Lasso, even when derived from analytical PDE solutions (i.e. without additional noise). Sparse regression te…
Robust physics discovery is of great interest for many scientific and engineering fields. Inspired by the principle that a representative model is the one simplest possible, a new model selection criteria considering both model's Parsimony and Sparsity is proposed. A Parsimony Enhanced Sparse Bayesian Learning (PeSBL) method is developed for discovering the governing Partial Differential Equations (PDEs) of nonlinear dynamical systems. Compared with the conventional Sparse Bayesian Learning (SBL) method, the PeSBL method promotes parsimony of the learned model in addition to its sparsity. In t…
The data-driven models allow one to define the model structure in cases when a priori information is not sufficient to build other types of models. The possible way to obtain physical interpretation is the data-driven differential equation discovery techniques. The existing methods of PDE (partial derivative equations) discovery are bound with the sparse regression. However, sparse regression is restricting the resulting model form, since the terms for PDE are defined before regression. The evolutionary approach described in the article has a symbolic regression as the background instead and t…
Joint discovery of governing partial differential equations from multi-source datasets by competitive optimization
Zhejiang Key Laboratory of Industrial Intelligence and Digital Twin, Eastern Institute of Technology（浙江工业智能与数字孪生重点实验室，东方理工高等研究院）；Department of Electrical Engineering, Tsinghua University（清华大学电机工程系）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工高等研究院宁波数字孪生研究院）
San José State University（圣何塞州立大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Universitá degli Studi di Napoli Federico II（那不勒斯费德里科二世大学）；Scuola Superiore Meridionale（南方高等学校）
We propose a machine learning framework for the data-driven discovery of macroscopic chemotactic Partial Differential Equations (PDEs) -- and the closures that lead to them -- from high-fidelity, individual-based stochastic simulations of E.coli bacterial motility. The fine scale, detailed, hybrid (continuum - Monte Carlo) simulation model embodies the underlying biophysics, and its parameters are informed from experimental observations of individual cells. We exploit Automatic Relevance Determination (ARD) within a Gaussian Process framework for the identification of a parsimonious set of col…
Johns Hopkins University（约翰斯·霍普金斯大学）；Princeton University（普林斯顿大学）；University of Rostock（罗斯托克大学）；Universitá degli Studi di Napoli Federico II（那不勒斯费德里科二世大学）
Complex spatiotemporal dynamics of physicochemical processes are often modeled at a microscopic level (through e.g. atomistic, agent-based or lattice models) based on first principles. Some of these processes can also be successfully modeled at the macroscopic level using e.g. partial differential equations (PDEs) describing the evolution of the right few macroscopic observables (e.g. concentration and momentum fields). Deriving good macroscopic descriptions (the so-called "closure problem") is often a time-consuming process requiring deep understanding/intuition about the system of interest. …
How Low Can You Go? Active Learning for Sparse Model Discovery in the Ultra-Low-Data Limit
Department of Mechanical Engineering, University of Washington（华盛顿大学机械工程系）；NSF AI Institute in Dynamic Systems, University of Washington（华盛顿大学NSF动态系统人工智能研究所）；Department of Aeronautics, Imperial College London（伦敦帝国理工学院航空系）
Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）
In recent years, data-driven methods have been developed to learn dynamical systems and partial differential equations (PDE). The goal of such work is discovering unknown physics and the corresponding equations. However, prior to achieving this goal, major challenges remain to be resolved, including learning PDE under noisy data and limited discrete data. To overcome these challenges, in this work, a deep-learning based data-driven method, called DL-PDE, is developed to discover the governing PDEs of underlying physical processes. The DL-PDE method combines deep learning via neural networks an…
Discovering hidden partial differential equations (PDEs) and operators from data is an important topic at the frontier between machine learning and numerical analysis. This doctoral thesis introduces theoretical results and deep learning algorithms to learn Green's functions associated with linear partial differential equations and rigorously justify PDE learning techniques. A theoretically rigorous algorithm is derived to obtain a learning rate, which characterizes the amount of training data needed to approximately learn Green's functions associated with elliptic PDEs. The construction conne…
Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）
Data-driven methods have recently made great progress in the discovery of partial differential equations (PDEs) from spatial-temporal data. However, several challenges remain to be solved, including sparse noisy data, incomplete candidate library, and spatially- or temporally-varying coefficients. In this work, a new framework, which combines neural network, genetic algorithm and adaptive methods, is put forward to address all of these challenges simultaneously. In the framework, a trained neural network is utilized to calculate derivatives and generate a large amount of meta-data, which solve…
Dynamical systems are typically governed by a set of linear/nonlinear differential equations. Distilling the analytical form of these equations from very limited data remains intractable in many disciplines such as physics, biology, climate science, engineering and social science. To address this fundamental challenge, we propose a novel Physics-informed Spline Learning (PiSL) framework to discover parsimonious governing equations for nonlinear dynamics, based on sparsely sampled noisy data. The key concept is to (1) leverage splines to interpolate locally the dynamics, perform analytical diff…
What do data tell us about physics-and what don't they tell us? There has been a surge of interest in using machine learning models to discover governing physical laws such as differential equations from data, but current methods lack uncertainty quantification to communicate their credibility. This work addresses this shortcoming from a Bayesian perspective. We introduce a novel model comprising "leaf" modules that learn to represent distinct experiments' spatiotemporal functional data as neural networks and a single "root" module that expresses a nonparametric distribution over their governi…
Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）；Peng Cheng Laboratory（鹏城实验室）
Data-driven discovery of partial differential equations (PDEs) has achieved considerable development in recent years. Several aspects of problems have been resolved by sparse regression-based and neural network-based methods. However, the performances of existing methods lack stability when dealing with complex situations, including sparse data with high noise, high-order derivatives and shock waves, which bring obstacles to calculating derivatives accurately. Therefore, a robust PDE discovery framework, called the robust deep learning-genetic algorithm (R-DLGA), that incorporates the physics-…
Symbolic Graph Networks for Robust PDE Discovery from Noisy Sparse Data
School of Software Engineering, Chengdu University of Information Technology（成都信息科技学院软件工程学院）；School of Statistics, Chengdu University of Information Technology（成都信息科技学院统计学院）；College of Applied Mathematics, Chengdu University of Information Technology（成都信息科技学院应用数学学院）；Key Laboratory of Mathematical Meteorology, Chengdu University of Information Technology（成都信息科技学院数学气象重点实验室）
Robust Moment Identification for Nonlinear PDEs via a Neural ODE Approach
Department of Mathematics and Statistics, University of Massachusetts Amherst（数学与统计学系，马萨诸塞大学阿默斯特分校）；School of Mathematics, Georgia Institute of Technology（数学学院，佐治亚理工学院）
