Physics informed neural networks (PINNs) have recently been very successfully applied for efficiently approximating inverse problems for PDEs. We focus on a particular class of inverse problems, the so-called data assimilation or unique continuation problems, and prove rigorous estimates on the generalization error of PINNs approximating them. An abstract framework is presented and conditional stability estimates for the underlying inverse problem are employed to derive the estimate on the PINN generalization error, providing rigorous justification for the use of PINNs in this context. The abstract framework is illustrated with examples of four prototypical linear PDEs. Numerical experiments, validating the proposed theory, are also presented.
Physics informed neural networks approximate solutions of PDEs by minimizing pointwise residuals. We derive rigorous bounds on the error, incurred by PINNs in approximating the solutions of a large class of linear parabolic PDEs, namely Kolmogorov equations that include the heat equation and Black-Scholes equation of option pricing, as examples. We construct neural networks, whose PINN residual (generalization error) can be made as small as desired. We also prove that the total $L^2$-error can be bounded by the generalization error, which in turn is bounded in terms of the training error, prov…
Laboratório Nacional de Computação Científica (LNCC)（国家科学计算实验室（LNCC））
Physics-Informed Neural Networks (PINNs) are machine learning tools that approximate the solution of general partial differential equations (PDEs) by adding them in some form as terms of the loss/cost function of a Neural Network. Most pieces of work in the area of PINNs tackle non-linear PDEs. Nevertheless, many interesting problems involving linear PDEs may benefit from PINNs; these include parametric studies, multi-query problems, and parabolic (transient) PDEs. The purpose of this paper is to explore PINNs for linear PDEs whose solutions may present one or more boundary layers. More specif…
Physics-informed neural networks (PINNs) as a means of discretizing partial differential equations (PDEs) are garnering much attention in the Computational Science and Engineering (CS&E) world. At least two challenges exist for PINNs at present: an understanding of accuracy and convergence characteristics with respect to tunable parameters and identification of optimization strategies that make PINNs as efficient as other computational science tools. The cost of PINNs training remains a major challenge of Physics-informed Machine Learning (PiML) - and, in fact, machine learning (ML) in general…
High precision PINNs in unbounded domains: application to singularity formulation in PDEs
California Institute of Technology（加州理工学院）；Massachusetts Institute of Technology（麻省理工学院）；The NSF Institute for Artificial Intelligence and Fundamental Interactions（NSF人工智能与基础相互作用研究所）
Physics informed neural networks (PINNs) are deep learning based techniques for solving partial differential equations (PDEs) encounted in computational science and engineering. Guided by data and physical laws, PINNs find a neural network that approximates the solution to a system of PDEs. Such a neural network is obtained by minimizing a loss function in which any prior knowledge of PDEs and data are encoded. Despite its remarkable empirical success in one, two or three dimensional problems, there is little theoretical justification for PINNs. As the number of data grows, PINNs generate a se…
Physics informed neural networks (PINNs) require regularity of solutions of the underlying PDE to guarantee accurate approximation. Consequently, they may fail at approximating discontinuous solutions of PDEs such as nonlinear hyperbolic equations. To ameliorate this, we propose a novel variant of PINNs, termed as weak PINNs (wPINNs) for accurate approximation of entropy solutions of scalar conservation laws. wPINNs are based on approximating the solution of a min-max optimization problem for a residual, defined in terms of Kruzkhov entropies, to determine parameters for the neural networks ap…
Compared with conventional numerical approaches to solving partial differential equations (PDEs), physics-informed neural networks (PINN) have manifested the capability to save development effort and computational cost, especially in scenarios of reconstructing the physics field and solving the inverse problem. Considering the advantages of parameter sharing, spatial feature extraction and low inference cost, convolutional neural networks (CNN) are increasingly used in PINN. However, some challenges still remain as follows. To adapt convolutional PINN to solve different PDEs, considerable effo…
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；Shandong University of Finance and Economics（山东财经大学）
We propose, Monte Carlo Nonlocal physics-informed neural networks (MC-Nonlocal-PINNs), which is a generalization of MC-fPINNs in \cite{guo2022monte}, for solving general nonlocal models such as integral equations and nonlocal PDEs. Similar as in MC-fPINNs, our MC-Nonlocal-PINNs handle the nonlocal operators in a Monte Carlo way, resulting in a very stable approach for high dimensional problems. We present a variety of test problems, including high dimensional Volterra type integral equations, hypersingular integral equations and nonlocal PDEs, to demonstrate the effectiveness of our approach.
Deflation-PINNs: Learning Multiple Solutions for PDEs and Landau-de Gennes
Department of Mathematics, Indian Institute of Science, Bangalore（印度科学研究所数学系，班加罗尔）；Department of Computing and Mathematical Sciences, California Institute of Technology（加州理工学院计算与数学科学系）
Dalian University of Technology（大连理工大学）；Harbin Institute of Technology（哈尔滨工业大学）；JD Explore Academy（京东探索研究院）
Physics-informed neural networks (PINNs) provide a deep learning framework for numerically solving partial differential equations (PDEs), and have been widely used in a variety of PDE problems. However, there still remain some challenges in the application of PINNs: 1) the mechanism of PINNs is unsuitable (at least cannot be directly applied) to exploiting a small size of (usually very few) extra informative samples to refine the networks; and 2) the efficiency of training PINNs often becomes low for some complicated PDEs. In this paper, we propose the generative adversarial physics-informed n…
School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai 200240, China（上海交通大学数学科学学院）；Department of Mathematics, University of Massachusetts Dartmouth, North Dartmouth, MA 02747（大学马萨诸塞州达特茅斯分校数学系）；Department of Chemical and Biomolecular Engineering, University of Pennsylvania, Philadelphia, PA 19104, United States of America（宾夕法尼亚大学化学与生物分子工程系）
CP-PINNs: Data-Driven Changepoints Detection in PDEs Using Online Optimized Physics-Informed Neural Networks
Stony Brook University（石溪大学）；Center of Excellence in Wireless and Information Technology（无线与信息技术卓越中心）；Institute for Advanced Computational Science（高级计算科学研究所）
We consider the approximation of a class of dynamic partial differential equations (PDE) of second order in time by the physics-informed neural network (PINN) approach, and provide an error analysis of PINN for the wave equation, the Sine-Gordon equation and the linear elastodynamic equation. Our analyses show that, with feed-forward neural networks having two hidden layers and the $\tanh$ activation function, the PINN approximation errors for the solution field, its time derivative and its gradient field can be effectively bounded by the training loss and the number of training data points (q…
Physics-informed Neural Networks (PINNs) have been widely used to obtain accurate neural surrogates for a system of Partial Differential Equations (PDE). One of the major limitations of PINNs is that the neural solutions are challenging to interpret, and are often treated as black-box solvers. While Symbolic Regression (SR) has been studied extensively, very few works exist which generate analytical expressions to directly perform SR for a system of PDEs. In this work, we introduce an end-to-end framework for obtaining mathematical expressions for solutions of PDEs. We use a trained PINN to ge…
Defense Innovation Institute, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）；Human Resource Center（人力资源中心）
Learning solutions of partial differential equations (PDEs) with Physics-Informed Neural Networks (PINNs) is an attractive alternative approach to traditional solvers due to its flexibility and ease of incorporating observed data. Despite the success of PINNs in accurately solving a wide variety of PDEs, the method still requires improvements in terms of computational efficiency. One possible improvement idea is to optimize the generation of training point sets. Residual-based adaptive sampling and quasi-uniform sampling approaches have been each applied to improve the training effects of PINN…
ETH Zürich（苏黎世联邦理工学院）；ETH AI Center（苏黎世联邦理工学院人工智能中心）
We propose a very general framework for deriving rigorous bounds on the approximation error for physics-informed neural networks (PINNs) and operator learning architectures such as DeepONets and FNOs as well as for physics-informed operator learning. These bounds guarantee that PINNs and (physics-informed) DeepONets or FNOs will efficiently approximate the underlying solution or solution operator of generic partial differential equations (PDEs). Our framework utilizes existing neural network approximation results to obtain bounds on more involved learning architectures for PDEs. We illustrate …
University of Bonn（波恩大学）；Delft University of Technology（代尔夫特理工大学）；RWTH Aachen University（亚琛工业大学）
Partial Differential Equations (PDEs) are notoriously difficult to solve. In general, closed-form solutions are not available and numerical approximation schemes are computationally expensive. In this paper, we propose to approach the solution of PDEs based on a novel technique that combines the advantages of two recently emerging machine learning based approaches. First, physics-informed neural networks (PINNs) learn continuous solutions of PDEs and can be trained with little to no ground truth data. However, PINNs do not generalize well to unseen domains. Second, convolutional neural network…
Shanghai Normal University（上海师范大学）；Tongji University（同济大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
We introduce a sampling based machine learning approach, Monte Carlo physics informed neural networks (MC-PINNs), for solving forward and inverse fractional partial differential equations (FPDEs). As a generalization of physics informed neural networks (PINNs), our method relies on deep neural network surrogates in addition to a stochastic approximation strategy for computing the fractional derivatives of the DNN outputs. A key ingredient in our MC-PINNs is to construct an unbiased estimation of the physical soft constraints in the loss function. Our directly sampling approach can yield less o…
Neural Measures for learning distributions of Random PDEs
DMAM, University of Crete, Greece（DMAM，希腊克里特大学）；IACM, Foundation for Research and Technology - Hellas, Greece（IACM，希腊基础研究与技术研究所）；MPS, University of Sussex, United Kingdom（MPS，英国 Sussex 大学）
Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time-space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposin…
National Innovation Institute of Defense Technology, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）
Physics-informed neural networks (PINNs) have been widely used to solve various scientific computing problems. However, large training costs limit PINNs for some real-time applications. Although some works have been proposed to improve the training efficiency of PINNs, few consider the influence of initialization. To this end, we propose a New Reptile initialization based Physics-Informed Neural Network (NRPINN). The original Reptile algorithm is a meta-learning initialization method based on labeled data. PINNs can be trained with less labeled data or even without any labeled data by adding p…
As an emerging technology in deep learning, physics-informed neural networks (PINNs) have been widely used to solve various partial differential equations (PDEs) in engineering. However, PDEs based on practical considerations contain multiple physical quantities and complex initial boundary conditions, thus PINNs often returns incorrect results. Here we take heat transfer problem in multilayer fabrics as a typical example. It is coupled by multiple temperature fields with strong correlation, and the values of variables are extremely unbalanced among different dimensions. We clarify the potenti…
Error Bounds for Physics-Informed Neural Networks in Fokker-Planck PDEs
Dept. of Aerospace Eng. Sciences University of Colorado Boulder（航空航天工程科学系，科罗拉多大学博尔德分校）；Center for Systems and Control Delft University of Technology（系统与控制中心，代尔夫特理工大学）
TINNs: Time-Induced Neural Networks for Solving Time-Dependent PDEs
Department of Applied Mathematics, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan（应用数学系，国立阳明交通大学，新竹30010，台湾）；Institute of Artificial Intelligence Innovation, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan（人工智能创新研究所，国立阳明交通大学，新竹30010，台湾）；National Center for Theoretical Sciences, National Taiwan University, Taipei 10617, Taiwan（理论科学研究中心，国立台湾大学，台北10617，台湾）
FO-PINNs: A First-Order formulation for Physics Informed Neural Networks
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NVIDIA（英伟达）；University of California Davis（加州大学戴维斯分校）；Illinois Institute of Technology（伊利诺伊理工学院）
A Physics Informed Neural Network (PINN) Methodology for Coupled Moving Boundary PDEs
Centre for Machine Intelligence and Data Science, Indian Institute of Technology Bombay（印度理工学院孟买分校机器智能与数据科学中心）；Department of Mechanical Engineering, Indian Institute of Technology Bombay（印度理工学院孟买分校机械工程系）
Tackling the Curse of Dimensionality in Fractional and Tempered Fractional PDEs with Physics-Informed Neural Networks
National University of Singapore（新加坡国立大学）；Worcester Polytechnic Institute（伍斯特理工学院）；Brown University（布朗大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
本文将 MC-fPINN 扩展为 MC-tfPINN，并用一维 Gaussian 求积替代 Monte Carlo 采样，在高达 100,000 维的分数阶与缓增分数阶 PDE 正、反问题中提升了精度和收敛速度。
A physics-informed neural network framework for modeling obstacle-related equations
African Institute for Mathematical Sciences（非洲数学科学研究所）；Technical University Berlin（柏林工业大学）；German Aerospace Center (DLR)（德国航空航天中心）；Quantum leap Africa（非洲量子飞跃组织）
Brown University（布朗大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiation. The PINN algorithm is simple, and it can be applied to different types of PDEs, including integro-differential equations, fractional PDEs, and stochastic PDEs. Moreover, from the implementation point of view, PINNs solve inverse problems as easily as forward problems. We pro…
