Physics informed neural networks (PINNs) are deep learning based techniques for solving partial differential equations (PDEs) encounted in computational science and engineering. Guided by data and physical laws, PINNs find a neural network that approximates the solution to a system of PDEs. Such a neural network is obtained by minimizing a loss function in which any prior knowledge of PDEs and data are encoded. Despite its remarkable empirical success in one, two or three dimensional problems, there is little theoretical justification for PINNs.
As the number of data grows, PINNs generate a sequence of minimizers which correspond to a sequence of neural networks. We want to answer the question: Does the sequence of minimizers converge to the solution to the PDE? We consider two classes of PDEs: linear second-order elliptic and parabolic. By adapting the Schauder approach and the maximum principle, we show that the sequence of minimizers strongly converges to the PDE solution in $C^0$. Furthermore, we show that if each minimizer satisfies the initial/boundary conditions, the convergence mode becomes $H^1$. Computational examples are provided to illustrate our theoretical findings. To the best of our knowledge, this is the first theoretical work that shows the consistency of PINNs.
Time-dependent Partial Differential Equations with given initial conditions are considered in this paper. New differentiation techniques of the unknown solution with respect to time variable are proposed. It is shown that the proposed techniques allow to generate accurate higher order derivatives simultaneously for a set of spatial points. The calculated derivatives can then be used for data-driven solution in different ways. An application for Physics Informed Neural Networks by the well-known DeepXDE software solution in Python under Tensorflow background framework has been presented for thr…
Physics informed neural networks approximate solutions of PDEs by minimizing pointwise residuals. We derive rigorous bounds on the error, incurred by PINNs in approximating the solutions of a large class of linear parabolic PDEs, namely Kolmogorov equations that include the heat equation and Black-Scholes equation of option pricing, as examples. We construct neural networks, whose PINN residual (generalization error) can be made as small as desired. We also prove that the total $L^2$-error can be bounded by the generalization error, which in turn is bounded in terms of the training error, prov…
ETH Zürich（苏黎世联邦理工学院）；ETH AI Center（苏黎世联邦理工学院人工智能中心）
We propose a very general framework for deriving rigorous bounds on the approximation error for physics-informed neural networks (PINNs) and operator learning architectures such as DeepONets and FNOs as well as for physics-informed operator learning. These bounds guarantee that PINNs and (physics-informed) DeepONets or FNOs will efficiently approximate the underlying solution or solution operator of generic partial differential equations (PDEs). Our framework utilizes existing neural network approximation results to obtain bounds on more involved learning architectures for PDEs. We illustrate …
HEATNETs: Explainable Random Feature Neural Networks for High-Dimensional Parabolic PDEs
Modelling and Engineering Risk and Complexity, Scuola Superiore Meridionale（南方高等学院建模与工程风险与复杂性研究中心）；Hopkins Extreme Materials Institute and Dept. of Chemical and Biomolecular Engineering, Johns Hopkins University（约翰斯·霍普金斯大学霍普金斯极端材料研究所和化学与生物分子工程系）；Department of Statistics and Stochastic Modelling and Applications Laboratory, Athens University of Economics and Business（雅典经济与商业大学统计学与随机建模与应用实验室）
Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks
Tsinghua University（清华大学）；Bauhaus-Universität Weimar（魏玛包豪斯大学）；CNPC Engineering Technology RD Company Limited（中国石油集团工程技术研究院有限公司）；Queensland University of Technology（昆士兰科技大学）；Leibniz University Hannover（汉诺威莱布尼茨大学）
Parameterized Physics-informed Neural Networks for Parameterized PDEs
Yonsei University（延世大学）；Arizona State University（亚利桑那州立大学）；LG CNS；Texas A&M University-Commerce（德州农工大学Commerce分校）；Oregon State University（俄勒冈州立大学）；KAIST（韩国科学技术院）
Northeastern University（东北大学）；University of Notre Dame（圣母大学）；MIT（麻省理工学院）
Partial differential equations (PDEs) play a fundamental role in modeling and simulating problems across a wide range of disciplines. Recent advances in deep learning have shown the great potential of physics-informed neural networks (PINNs) to solve PDEs as a basis for data-driven modeling and inverse analysis. However, the majority of existing PINN methods, based on fully-connected NNs, pose intrinsic limitations to low-dimensional spatiotemporal parameterizations. Moreover, since the initial/boundary conditions (I/BCs) are softly imposed via penalty, the solution quality heavily relies on h…
Physics-informed neural networks (PINNs) as a means of discretizing partial differential equations (PDEs) are garnering much attention in the Computational Science and Engineering (CS&E) world. At least two challenges exist for PINNs at present: an understanding of accuracy and convergence characteristics with respect to tunable parameters and identification of optimization strategies that make PINNs as efficient as other computational science tools. The cost of PINNs training remains a major challenge of Physics-informed Machine Learning (PiML) - and, in fact, machine learning (ML) in general…
Quantum-classical physics-informed Kolmogorov-Arnold networks for PDEs
School of Petroleum Engineering, Yangtze University（扬州大学石油工程学院）；School of Computer Science, Yangtze University（扬州大学计算机科学学院）；State Key Laboratory of Low Carbon Catalysis and Carbon Dioxide Utilization (Yangtze University)（扬州大学低碳催化与二氧化碳利用国家重点实验室）；Western Research Institute, Yangtze University（扬州大学西部研究院）
QCPINN: Quantum-Classical Physics-Informed Neural Networks for Solving PDEs
Computational Science and Engineering Department, Informatics Institute Istanbul Technical University（信息学院理工学院计算科学与工程系）；Scientific Computing, Rutherford Appleton Laboratory Science and Technology Facilities Council (STFC)（科学与技术设施委员会（STFC）科学计算部）
There has been a growing interest in the use of Deep Neural Networks (DNNs) to solve Partial Differential Equations (PDEs). Despite the promise that such approaches hold, there are various aspects where they could be improved. Two such shortcomings are (i) their computational inefficiency relative to classical numerical methods, and (ii) the non-interpretability of a trained DNN model. In this work we present ASPINN, an anisotropic extension of our earlier work called SPINN--Sparse, Physics-informed, and Interpretable Neural Networks--to solve PDEs that addresses both these issues. ASPINNs gen…
This paper addresses distributional offline continuous-time reinforcement learning (DOCTR-L) with stochastic policies for high-dimensional optimal control. A soft distributional version of the classical Hamilton-Jacobi-Bellman (HJB) equation is given by a semilinear partial differential equation (PDE). This `soft HJB equation' can be learned from offline data without assuming that the latter correspond to a previous optimal or near-optimal policy. A data-driven solution of the soft HJB equation uses methods of Neural PDEs and Physics-Informed Neural Networks developed in the field of Scientifi…
University of Bonn（波恩大学）；Delft University of Technology（代尔夫特理工大学）；RWTH Aachen University（亚琛工业大学）
Partial Differential Equations (PDEs) are notoriously difficult to solve. In general, closed-form solutions are not available and numerical approximation schemes are computationally expensive. In this paper, we propose to approach the solution of PDEs based on a novel technique that combines the advantages of two recently emerging machine learning based approaches. First, physics-informed neural networks (PINNs) learn continuous solutions of PDEs and can be trained with little to no ground truth data. However, PINNs do not generalize well to unseen domains. Second, convolutional neural network…
We investigate the resolution of parabolic PDEs via Extreme Learning Machine (ELMs) Neural Networks, which have a single hidden layer and can be trained at a modest computational cost as compared with Deep Learning Neural Networks. Our approach addresses the time evolution by applying classical ODEs techniques and uses ELM-based collocation for solving the resulting stationary elliptic problems. In this framework, the $θ$-method and Backward Difference Formulae (BDF) techniques are investigated on some linear parabolic PDEs that are challeging problems for the stability and accuracy properties…
We propose a novel algorithm, based on physics-informed neural networks (PINNs) to efficiently approximate solutions of nonlinear dispersive PDEs such as the KdV-Kawahara, Camassa-Holm and Benjamin-Ono equations. The stability of solutions of these dispersive PDEs is leveraged to prove rigorous bounds on the resulting error. We present several numerical experiments to demonstrate that PINNs can approximate solutions of these dispersive PDEs very accurately
How to build an accurate reduced order model (ROM) for multidimensional time dependent partial differential equations (PDEs) is quite open. In this paper, we propose a new ROM for linear parabolic PDEs. We prove that our new method can be orders of magnitude faster than standard solvers, and is also much less memory intensive. Under some assumptions on the problem data, we prove that the convergence rates of the new method is the same with standard solvers. Numerical experiments are presented to confirm our theoretical result.
In [2019, Space-time least-squares finite elements for parabolic equations, arXiv:1911.01942] by Führer& Karkulik, well-posedness of a space-time First-Order System Least-Squares formulation of the heat equation was proven. In the present work, this result is generalized to general second order parabolic PDEs with possibly inhomogenoeus boundary conditions, and plain convergence of a standard adaptive finite element method driven by the least-squares estimator is demonstrated. The proof of the latter easily extends to a large class of least-squares formulations.
ETH Zurich（苏黎世联邦理工学院）；University of Münster（明斯特大学）；Nanyang Technological University（南洋理工大学）
In this paper we introduce a numerical method for nonlinear parabolic PDEs that combines operator splitting with deep learning. It divides the PDE approximation problem into a sequence of separate learning problems. Since the computational graph for each of the subproblems is comparatively small, the approach can handle extremely high-dimensional PDEs. We test the method on different examples from physics, stochastic control and mathematical finance. In all cases, it yields very good results in up to 10,000 dimensions with short run times.
We study a general class of singular degenerate parabolic stochastic partial differential equations (SPDEs) which include, in particular, the stochastic porous medium equations and the stochastic fast diffusion equation. We propose a fully discrete numerical approximation of the considered SPDEs based on the very weak formulation. By exploiting the monotonicity properties of the proposed formulation we prove the convergence of the numerical approximation towards the unique solution. Furthermore, we construct an implementable finite element scheme for the spatial discretization of the very weak…
In this paper, we employed linear algebra and functional analysis to determine necessary and sufficient conditions for oscillation-free and stable solutions to linear and nonlinear parabolic partial differential equations. We applied singular value decomposition and Fourier analysis to various finite difference schemes to extract patterns in the eigenfunctions (sampled by the eigenvectors) and the shape of their eigenspectrum. With these, we determined how the initial and boundary conditions affect the frequency and long term behavior of numerical oscillations, as well as the location of solut…
This paper considers optimal control of fractional parabolic PDEs with both state and control constraints. The key challenge is how to handle the state constraints. Similarly, to the elliptic case, in this paper, we establish several new mathematical tools in the parabolic setting that are of wider interest. For example, existence of solution to the fractional parabolic equation with measure data on the right-hand-side. We employ the Moreau-Yosida regularization to handle the state constraints. We establish convergence, with rate, of the regularized optimal control problem to the original one.…
We consider the discretization of elliptic boundary-value problems by variational physics-informed neural networks (VPINNs), in which test functions are continuous, piecewise linear functions on a triangulation of the domain. We define an a posteriori error estimator, made of a residual-type term, a loss-function term, and data oscillation terms. We prove that the estimator is both reliable and efficient in controlling the energy norm of the error between the exact and VPINN solutions. Numerical results are in excellent agreement with the theoretical predictions.
This paper is concerned with the regularity of solutions to parabolic evolution equations. Special attention is paid to the smoothness in the specific anisotropic scale $\ B^{r\mathbf{a}}_{τ,τ}, \ \frac{1}τ=\frac{r}{d}+\frac{1}{p}\ $ of Besov spaces where $\mathbf{a}$ measures the anisotropy. The regularity in these spaces determines the approximation order that can be achieved by fully space-time adaptive approximation schemes. In particular, we show that for the heat equation our results significantly improve previous results by Aimar and Gomez [3].
Freie Universität Berlin（柏林自由大学）；BTU Cottbus-Senftenberg（科特布斯-森夫滕贝格勃兰登堡工业大学）；dida Datenschmiede GmbH（dida数据锻造有限公司）；Technische Universität Berlin（柏林工业大学）；Universität Potsdam（波茨坦大学）
High-dimensional partial differential equations (PDEs) are ubiquitous in economics, science and engineering. However, their numerical treatment poses formidable challenges since traditional grid-based methods tend to be frustrated by the curse of dimensionality. In this paper, we argue that tensor trains provide an appealing approximation framework for parabolic PDEs: the combination of reformulations in terms of backward stochastic differential equations and regression-type methods in the tensor format holds the promise of leveraging latent low-rank structures enabling both compression and ef…
In this paper, we propose forward and backward stochastic differential equations (FBSDEs) based deep neural network (DNN) learning algorithms for the solution of high dimensional quasilinear parabolic partial differential equations (PDEs), which are related to the FBSDEs by the Pardoux-Peng theory. The algorithms rely on a learning process by minimizing the pathwise difference between two discrete stochastic processes, defined by the time discretization of the FBSDEs and the DNN representation of the PDE solutions, respectively. The proposed algorithms are shown to generate DNN solutions for a…
The tangential condition was introduced in [Hanke et al., 95] as a sufficient condition for convergence of the Landweber iteration for solving ill-posed problems. In this paper we present a series of time dependent benchmark inverse problems for which we can verify this condition.
Explicit stabilized integrators are an efficient alternative to implicit or semi-implicit methods to avoid the severe timestep restriction faced by standard explicit integrators applied to stiff diffusion problems. In this paper, we provide a fully discrete strong convergence analysis of a family of explicit stabilized methods coupled with finite element methods for a class of parabolic semilinear deterministic and stochastic partial differential equations. Numerical experiments including the semilinear stochastic heat equation with space-time white noise confirm the theoretical findings.
We propose an approximation scheme for a class of semilinear parabolic equations that are convex and coercive in their gradients. Such equations arise often in pricing and portfolio management in incomplete markets and, more broadly, are directly connected to the representation of solutions to backward stochastic differential equations. The proposed scheme is based on splitting the equation in two parts, the first corresponding to a linear parabolic equation and the second to a Hamilton-Jacobi equation. The solutions of these two equations are approximated using, respectively, the Feynman-Kac …
The goal of this work is to develop deep learning numerical methods for solving option XVA pricing problems given by non-linear PDE models. A novel strategy for the treatment of the boundary conditions is proposed, which allows to get rid of the heuristic choice of the weights for the different addends that appear in the loss function related to the training process. It is based on defining the losses associated to the boundaries by means of the PDEs that arise from substituting the related conditions into the model equation itself. Further, automatic differentiation is employed to obtain accu…
We consider the design of structure-preserving discretization methods for the solution of systems of boundary controlled Partial Differential Equations (PDEs) thanks to the port-Hamiltonian formalism. We first provide a novel general structure of infinite-dimensional port-Hamiltonian systems (pHs) for which the Partitioned Finite Element Method (PFEM) straightforwardly applies. The proposed strategy is applied to abstract multidimensional linear hyperbolic and parabolic systems of PDEs. Then we show that instructional model problems based on the wave equation, Mindlin equation and heat equatio…
