Recent works have shown that deep neural networks can be employed to solve partial differential equations, giving rise to the framework of physics informed neural networks. We introduce a generalization for these methods that manifests as a scaling parameter which balances the relative importance of the different constraints imposed by partial differential equations. A mathematical motivation of these generalized methods is provided, which shows that for linear and well-posed partial differential equations, the functional form is convex. We then derive a choice for the scaling parameter that is optimal with respect to a measure of relative error. Because this optimal choice relies on having full knowledge of analytical solutions, we also propose a heuristic method to approximate this optimal choice. The proposed methods are compared numerically to the original methods on a variety of model partial differential equations, with the number of data points being updated adaptively. For several problems, including high-dimensional PDEs the proposed methods are shown to significantly enhance accuracy.
The paper introduces a very simple and fast computation method for high-dimensional integrals to solve high-dimensional Kolmogorov partial differential equations (PDEs). The new machine learning-based method is obtained by solving a stochastic weighted minimization with stochastic gradient descent which is inspired by a high-order weak approximation scheme for stochastic differential equations (SDEs) with Malliavin weights. Then solutions to high-dimensional Kolmogorov PDEs or expectations of functionals of solutions to high-dimensional SDEs are accurately approximated without suffering from t…
Variable steps implicit-explicit multistep methods for PDEs have been presented in [17], where the zero-stability is studied for ODEs; however, the stability analysis still remains an open question for PDEs. Based on the idea of linear multistep methods, we present a simple weighted and shifted BDF2 methods with variable steps for the parabolic problems, which serve as a bridge between BDF2 and Crank-Nicolson scheme. The contributions of this paper are as follows: we first prove that the optimal adjacent time-step ratios for the weighted and shifted BDF2, which greatly improve the maximum time…
The quantification of multivariate uncertainties in partial differential equations can easily exceed any computing capacity unless proper measures are taken to reduce the complexity of the model. In this work, we propose a multidimensional Galerkin Proper Orthogonal Decomposition that optimally reduces each dimension of a tensorized product space. We provide the analytical framework and results that define and quantify the low-dimensional approximation. We illustrate its application for uncertainty modeling with Polynomial Chaos Expansions and show its efficiency in a numerical example.
We explore the possibility for using boundary data to identify sources in elliptic PDEs. Even though the associated forward operator has a large null space, it turns out that box constraints, combined with weighted sparsity regularization, can enable rather accurate recovery of sources with constant magnitude/strength. In addition, for sources with varying strength, the support of the inverse solution will be a subset of the support of the true source. We present both an analysis of the problem and a series of numerical experiments. Our work only addresses discretized problems. The reason for …
Iowa State University（爱荷华州立大学）；New York University（纽约大学）
We consider a mesh-based approach for training a neural network to produce field predictions of solutions to parametric partial differential equations (PDEs). This approach contrasts current approaches for "neural PDE solvers" that employ collocation-based methods to make point-wise predictions of solutions to PDEs. This approach has the advantage of naturally enforcing different boundary conditions as well as ease of invoking well-developed PDE theory -- including analysis of numerical stability and convergence -- to obtain capacity bounds for our proposed neural networks in discretized domai…
Physics Informed Neural Networks (PINNs) have frequently been used for the numerical approximation of Partial Differential Equations (PDEs). The goal of this paper is to construct PINNs along with a computable upper bound of the error, which is particularly relevant for model reduction of Parameterized PDEs (PPDEs). To this end, we suggest to use a weighted sum of expansion coefficients of the residual in terms of an adaptive wavelet expansion both for the loss function and an error bound. This approach is shown here for elliptic PPDEs using both the standard variational and an optimally stabl…
We obtained convergence rates of the collocation approximation by deep ReLU neural networks of solutions to elliptic PDEs with lognormal inputs, parametrized by $\boldsymbol{y}$ from the non-compact set $\mathbb{R}^\infty$. The approximation error is measured in the norm of the Bochner space $L_2(\mathbb{R}^\infty, V, γ)$, where $γ$ is the infinite tensor product standard Gaussian probability measure on $\mathbb{R}^\infty$ and $V$ is the energy space. We also obtained similar results for the case when the lognormal inputs are parametrized on $\mathbb{R}^M$ with very large dimension $M$, and th…
We propose and analyse numerical algorithms based on weighted least squares for the approximation of a real-valued function on a general bounded domain $Ω\subset \mathbb{R}^d$. Given any $n$-dimensional approximation space $V_n \subset L^2(Ω)$, the analysis in [6] shows the existence of stable and optimally converging weighted least-squares estimators, using a number of function evaluations $m$ of the order $n \log n$. When an $L^2(Ω)$-orthonormal basis of $V_n$ is available in analytic form, such estimators can be constructed using the algorithms described in [6,Section 5]. If the basis also …
We investigate non-adaptive methods of deep ReLU neural network approximation in Bochner spaces $L_2({\mathbb U}^\infty, X, μ)$ of functions on ${\mathbb U}^\infty$ taking values in a separable Hilbert space $X$, where ${\mathbb U}^\infty$ is either ${\mathbb R}^\infty$ equipped with the standard Gaussian probability measure, or ${\mathbb I}^\infty:= [-1,1]^\infty$ equipped with the Jacobi probability measure. Functions to be approximated are assumed to satisfy a certain weighted $\ell_2$-summability of the generalized chaos polynomial expansion coefficients with respect to the measure $μ$. We…
This paper studies numerical methods for the approximation of elliptic PDEs with lognormal coefficients of the form $-{\rm div}(a\nabla u)=f$ where $a=\exp(b)$ and $b$ is a Gaussian random field. The approximant of the solution $u$ is an $n$-term polynomial expansion in the scalar Gaussian random variables that parametrize $b$. We present a general convergence analysis of weighted least-squares approximants for smooth and arbitrarily rough random field, using a suitable random design, for which we prove optimality in the following sense: their convergence rate matches exactly or closely the ra…
Sampling recovery in Bochner spaces and applications to parametric PDEs
Mathematisch-Geographische Fakultät, KU Eichstätt-Ingolstadt（凯泽大学艾希施泰特-因戈尔施塔特数学地理学院）；Information Technology Institute, Vietnam National University, Hanoi（越南国家大学河内信息技术研究所）
The recently developed physics-informed machine learning has made great progress for solving nonlinear partial differential equations (PDEs), however, it may fail to provide reasonable approximations to the PDEs with discontinuous solutions. In this paper, we focus on the discrete time physics-informed neural network (PINN), and propose a hybrid PINN scheme for the nonlinear PDEs. In this approach, the local solution structures are classified as smooth and nonsmooth scales by introducing a discontinuity indicator, and then the automatic differentiation technique is employed for resolving smoot…
Given any domain $X\subseteq \mathbb{R}^d$ and a probability measure $ρ$ on $X$, we study the problem of approximating in $L^2(X,ρ)$ a given function $u:X\to\mathbb{R}$, using its noiseless pointwise evaluations at random samples. For any given linear space $V\subset L^2(X,ρ)$ with dimension $n$, previous works have shown that stable and optimally converging Weighted Least-Squares (WLS) estimators can be constructed using $m$ random samples distributed according to an auxiliary probability measure $μ$ that depends on $V$, with $m$ being linearly proportional to $n$ up to a logarithmic term. As…
In this paper we extend results taken from compressed sensing to recover Hilbert-space valued vectors. This is an important problem in parametric function approximation in particular when the number of parameters is high. By expanding our target functions in a polynomial chaos and assuming some compressibility of such an expansion, we can exploit structured sparsity (typically a group sparsity structure) to recover the sequence of coefficients with high accuracy. While traditional compressed sensing would typically expect a number of snapshots scaling exponentially with the number of parameter…
Rigorous numerics for ill-posed PDEs: periodic orbits in the Boussinesq equation
VU University Amsterdam（阿姆斯特丹自由大学）；Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo（圣保罗大学数学与计算科学学院）；Université Laval, Département de Mathématiques et de Statistique（拉瓦尔大学数学与统计系）
Weighted approximate Fekete points: Sampling for least-squares polynomial approximation
Department of Mathematics, Shanghai Normal University（上海师范大学数学系）；Department of Mathematics and Scientific Computing and Imaging Institute, University of Utah（犹他大学数学与科学计算及成像研究所）；Department of Mathematics, Southeast University（东南大学数学系）；LSEC, Institute of Computational Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院计算数学与科学工程计算研究所）
We present a method for computing A-optimal sensor placements for infinite-dimensional Bayesian linear inverse problems governed by PDEs with irreducible model uncertainties. Here, irreducible uncertainties refers to uncertainties in the model that exist in addition to the parameters in the inverse problem, and that cannot be reduced through observations. Specifically, given a statistical distribution for the model uncertainties, we compute the optimal design that minimizes the expected value of the posterior covariance trace. The expected value is discretized using Monte Carlo leading to an o…
A weighted POD-reduction approach for parametrized PDE-constrained Optimal Control Problems with random inputs and applications to environmental sciences
We develop fast and scalable methods for computing reduced-order nonlinear solutions (RONS). RONS was recently proposed as a framework for reduced-order modeling of time-dependent partial differential equations (PDEs), where the modes depend nonlinearly on a set of time-varying parameters. RONS uses a set of ordinary differential equations (ODEs) for the parameters to optimally evolve the shape of the modes to adapt to the PDE's solution. This method has already proven extremely effective in tackling challenging problems such as advection-dominated flows and high-dimensional PDEs. However, as …
Leibniz University（莱布尼茨大学）；Leibniz University Hannover（汉诺威莱布尼茨大学）；Tongji University（同济大学）；Ton Duc Thang University（孙德胜大学）
Deep learning has shown successful application in visual recognition and certain artificial intelligence tasks. Deep learning is also considered as a powerful tool with high flexibility to approximate functions. In the present work, functions with desired properties are devised to approximate the solutions of PDEs. Our approach is based on a posteriori error estimation in which the adjoint problem is solved for the error localization to formulate an error estimator within the framework of neural network. An efficient and easy to implement algorithm is developed to obtain a posteriori error est…
Hessian-augmented Supervised Learning for Hamilton-Jacobi-Bellman PDEs
Department of Mathematics, Imperial College London, South Kensington Campus（帝国理工学院伦敦数学系，南肯辛顿校区）；Department of Mathematics and Statistics, University of Konstanz（康斯坦茨大学数学与统计学系）；RICAM and Institute of Mathematics and Scientific Computing, University of Graz（格拉茨大学RICAM与数学与计算科学研究所）
We address a new numerical scheme based on a class of machine learning methods, the so-called Extreme Learning Machines with both sigmoidal and radial-basis functions, for the computation of steady-state solutions and the construction of (one dimensional) bifurcation diagrams of nonlinear partial differential equations (PDEs). For our illustrations, we considered two benchmark problems, namely (a) the one-dimensional viscous Burgers with both homogeneous (Dirichlet) and non-homogeneous mixed boundary conditions, and, (b) the one and two-dimensional Liouville-Bratu-Gelfand PDEs with homogeneous…
In this paper, we address the task of setting up an optimal production plan taking into account an uncertain demand. The energy system is represented by a system of hyperbolic partial differential equations (PDEs) and the uncertain demand stream is captured by an Ornstein-Uhlenbeck process. We determine the optimal inflow depending on the producer's risk preferences. The resulting output is intended to optimally match the stochastic demand for the given risk criteria. We use uncertainty quantification for an adaptation to different levels of risk aversion. More precisely, we use two types of c…
PG-KINN: A Physics-Informed Petrov-Galerkin Kolmogorov-Arnold Network for Solving Forward and Inverse PDEs
Shahid Beheshti University（沙希德·贝赫什提大学）；Institute for Research in Fundamental Sciences (IPM)（基础科学研究所（IPM））；Cyberspace Research Institute, Shahid Beheshti University（沙希德·贝赫什提大学网络空间研究所）；University of Hertfordshire（赫特福德大学）
