Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditions
Faculty of Mathematics, University of Vienna（维也纳大学数学学院）；Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences（奥地利科学院约翰·拉东计算与应用数学研究所）
In recent work it has been established that deep neural networks are capable of approximating solutions to a large class of parabolic partial differential equations without incurring the curse of dimension. However, all this work has been restricted to problems formulated on the whole Euclidean domain. On the other hand, most problems in engineering and the sciences are formulated on finite domains and subjected to boundary conditions. The present paper considers an important such model problem, namely the Poisson equation on a domain $D\subset \mathbb{R}^d$ subject to Dirichlet boundary conditions. It is shown that deep neural networks are capable of representing solutions of that problem without incurring the curse of dimension. The proofs are based on a probabilistic representation of the solution to the Poisson equation as well as a suitable sampling method.
This paper studies an unsupervised deep learning-based numerical approach for solving partial differential equations (PDEs). The approach makes use of the deep neural network to approximate solutions of PDEs through the compositional construction and employs least-squares functionals as loss functions to determine parameters of the deep neural network. There are various least-squares functionals for a partial differential equation. This paper focuses on the so-called first-order system least-squares (FOSLS) functional studied in [3], which is based on a first-order system of scalar second-orde…
In this paper, we propose a Galerkin finite element method for the elliptic optimal control problem governed by the Riesz space-fractional PDEs on 2D domains with control variable being discretized by variational discretization technique. The optimality condition is derived and priori error estimates of control, costate and state variables are successfully established. Numerical test is carried out to illustrate the accuracy performance of this approach.
We formulate and analyze a goal-oriented adaptive finite element method (GOAFEM) for a semilinear elliptic PDE and a linear goal functional. The strategy involves the finite element solution of a linearized dual problem, where the linearization is part of the adaptive strategy. Linear convergence and optimal algebraic convergence rates are shown.
In this work, we derive a priori error estimate of the mixed residual method when solving some elliptic PDEs. Our work is the first theoretical study of this method. We prove that the neural network solutions will converge if we increase the training samples and network size without any constraint on the ratio of training samples to the network size. Besides, our results suggest that the mixed residual method can recover high order derivatives better than the deep Ritz method, which has also been verified by our numerical experiments.
In this paper, we perform the convergence analysis of unsupervised Legendre--Galerkin neural networks (ULGNet), a deep-learning-based numerical method for solving partial differential equations (PDEs). Unlike existing deep learning-based numerical methods for PDEs, the ULGNet expresses the solution as a spectral expansion with respect to the Legendre basis and predicts the coefficients with deep neural networks by solving a variational residual minimization problem. Since the corresponding loss function is equivalent to the residual induced by the linear algebraic system depending on the choic…
We propose a First-Order System Least Squares (FOSLS) method based on deep-learning for numerically solving second-order elliptic PDEs. The method we propose is capable of dealing with either variational and non-variational problems, and because of its meshless nature, it can also deal with problems posed in high-dimensional domains. We prove the $Γ$-convergence of the neural network approximation towards the solution of the continuous problem, and extend the convergence proof to some well-known related methods. Finally, we present several numerical examples illustrating the performance of our…
Physics informed neural networks (PINNs) are deep learning based techniques for solving partial differential equations (PDEs) encounted in computational science and engineering. Guided by data and physical laws, PINNs find a neural network that approximates the solution to a system of PDEs. Such a neural network is obtained by minimizing a loss function in which any prior knowledge of PDEs and data are encoded. Despite its remarkable empirical success in one, two or three dimensional problems, there is little theoretical justification for PINNs. As the number of data grows, PINNs generate a se…
We propose a data-driven approach to solve multiscale elliptic PDEs with random coefficients based on the intrinsic low dimension structure of the underlying elliptic differential operators. Our method consists of offline and online stages. At the offline stage, a low dimension space and its basis are extracted from the data to achieve significant dimension reduction in the solution space. At the online stage, the extracted basis will be used to solve a new multiscale elliptic PDE efficiently. The existence of low dimension structure is established by showing the high separability of the under…
Based on the Fourier extension, we propose an oversampling collocation method for solving the elliptic partial differential equations with variable coefficients over arbitrary irregular domains. This method only uses the function values on the equispaced nodes, which has low computational cost and versatility. While a variety of numerical experiments are presented to demonstrate the effectiveness of this method, it shows that the approximation error fast reaches a plateau with increasing the degrees of freedom, due to the inherent ill-conditioned of frames.
This paper establishes the optimal $H^1$-norm error estimate for a nonstandard finite element method for approximating $H^2$ strong solutions of second order linear elliptic PDEs in non-divergence form with continuous coefficients. To circumvent the difficulty of lacking an effective duality argument for this class of PDEs, a new analysis technique is introduced; the crux of it is to establish an $H^1$-norm stability estimate for the finite element approximation operator which mimics a similar estimate for the underlying PDE operator recently established by the authors and its proof is based o…
Duke University（杜克大学）；University of Massachusetts（马萨诸塞大学）
Numerical solutions to high-dimensional partial differential equations (PDEs) based on neural networks have seen exciting developments. This paper derives complexity estimates of the solutions of $d$-dimensional second-order elliptic PDEs in the Barron space, that is a set of functions admitting the integral of certain parametric ridge function against a probability measure on the parameters. We prove under some appropriate assumptions that if the coefficients and the source term of the elliptic PDE lie in Barron spaces, then the solution of the PDE is $ε$-close with respect to the $H^1$ norm …
In this paper, we present proofs of the coerciveness of first-order system least-squares methods for general (possibly indefinite) second-order linear elliptic PDEs under a minimal uniqueness assumption. For general linear second-order elliptic PDEs, the uniqueness, existence, and well-posedness are equivalent due to the compactness of the operator and Fredholm alternative. Thus only a minimal uniqueness assumption is assumed: the homogeneous equation has a unique zero solution. The coerciveness of the standard variational problem is not required. The paper's main contribution is our first pro…
In this paper, we study the stabilizer-free weak Galerkin methods on polytopal meshes for a class of second order elliptic boundary value problems of divergence form and with gradient nonlinearity in the principal coefficient. With certain assumptions on the nonlinear coefficient, we show that the discrete problem has a unique solution. This is achieved by showing that the associated operator satisfies certain continuity and monotonicity properties. With the help of these properties, we derive optimal error estimates in the energy norm. We present several numerical examples to verify the error…
This work is a follow-up to our previous contribution ("Convergence of sparse collocation for functions of countably many Gaussian random variables (with application to elliptic PDEs)", SIAM J. Numer. Anal., 2018), and contains further insights on some aspects of the solution of elliptic PDEs with lognormal diffusion coefficients using sparse grids. Specifically, we first focus on the choice of univariate interpolation rules, advocating the use of Gaussian Leja points as introduced by Narayan and Jakeman ("Adaptive Leja sparse grid constructions for stochastic collocation and high-dimensional …
We develop a boundary integral equation solver for elliptic partial differential equations on complex \threed geometries. Our method is efficient, high-order accurate and robustly handles complex geometries. A key component is our singular and near-singular layer potential evaluation scheme, \qbkix: a simple extrapolation of the solution along a line to the boundary. We present a series of geometry-processing algorithms required for \qbkix to run efficiently with accuracy guarantees on arbitrary geometries and an adaptive upsampling scheme based on a iteration-free heuristic for quadrature err…
We obtained convergence rates of the collocation approximation by deep ReLU neural networks of solutions to elliptic PDEs with lognormal inputs, parametrized by $\boldsymbol{y}$ from the non-compact set $\mathbb{R}^\infty$. The approximation error is measured in the norm of the Bochner space $L_2(\mathbb{R}^\infty, V, γ)$, where $γ$ is the infinite tensor product standard Gaussian probability measure on $\mathbb{R}^\infty$ and $V$ is the energy space. We also obtained similar results for the case when the lognormal inputs are parametrized on $\mathbb{R}^M$ with very large dimension $M$, and th…
Two new relaxation schemes are proposed for the smoothing step in the geometric multigrid solution of PDEs on 2D and 3D stretched structured grids. The new schemes are characterized by efficient line relaxation on branched sets of lines of alternating colour, where the lines are constructed to be everywhere orthogonal to the local direction of maximum grid clustering. Tweed relaxation is best suited for grid clustering near the boundaries of the computational domain, whereas wireframe relaxation is best suited for grid clustering near the centre of the computational domain. On strongly stretch…
In this paper we introduce an algorithm based on a sparse grid adaptive refinement, for the approximation of the eigensolutions to parametric problems arising from elliptic partial differential equations. In particular, we are interested in detecting the crossing of the hypersurfaces describing the eigenvalues as a function of the parameters. The a priori matching is followed by an a posteriori verification, driven by a suitably defined error indicator. At a given refinement level, a sparse grid approach is adopted for the construction of the grid of the next level, by using the marking given …
While deep learning has achieved remarkable success in solving partial differential equations (PDEs), it still faces significant challenges, particularly when the PDE solutions have low regularity or singularities. To address these issues, we propose the Weak TransNet (WTN) method, based on a Petrov-Galerkin formulation, for solving elliptic PDEs in this work, though its framework may extend to other classes of equations. Specifically, the neural feature space defined by TransNet (Zhang et al., 2023) is used as the trial space, while the test space is composed of radial basis functions. Since …
An EDG Method for Distributed Optimal Control of Elliptic PDEs
College of Mathematics, Sichuan University（四川大学数学学院）；Department of Mathematics and Statistics, Missouri University of Science and Technology（密苏里科技大学数学与统计系）
We propose and analyse a fully adaptive strategy for solving elliptic PDEs with random data in this work. A hierarchical sequence of adaptive mesh refinements for the spatial approximation is combined with adaptive anisotropic sparse Smolyak grids in the stochastic space in such a way as to minimize the computational cost. The novel aspect of our strategy is that the hierarchy of spatial approximations is sample dependent so that the computational effort at each collocation point can be optimised individually. We outline a rigorous analysis for the convergence and computational complexity of t…
Sparse polynomial approximation for optimal control problems constrained by elliptic PDEs with lognormal random coefficients
Oden Institute for Computational Engineering & Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校奥登计算工程与科学学院）；Department of Mechanical Engineering, and Department of Geological Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校机械工程系和地质科学系）
In this paper, we propose a certified reduced basis (RB) method for quasilinear elliptic problems together with its application to nonlinear magnetostatics equations, where the later model permanent magnet synchronous motors (PMSM). The parametrization enters through the geometry of the domain and thus, combined with the nonlinearity, drives our reduction problem. We provide a residual-based a-posteriori error bound which, together with the Greedy approach, allows to construct reduced-basis spaces of small dimensions. We use the empirical interpolation method (EIM) to guarantee the efficient o…
In this paper, we study the statistical limits of deep learning techniques for solving elliptic partial differential equations (PDEs) from random samples using the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs). To simplify the problem, we focus on a prototype elliptic PDE: the Schrödinger equation on a hypercube with zero Dirichlet boundary condition, which has wide application in the quantum-mechanical systems. We establish upper and lower bounds for both methods, which improves upon concurrently developed upper bounds for this problem via a fast rate generalization boun…
This paper studies numerical methods for the approximation of elliptic PDEs with lognormal coefficients of the form $-{\rm div}(a\nabla u)=f$ where $a=\exp(b)$ and $b$ is a Gaussian random field. The approximant of the solution $u$ is an $n$-term polynomial expansion in the scalar Gaussian random variables that parametrize $b$. We present a general convergence analysis of weighted least-squares approximants for smooth and arbitrarily rough random field, using a suitable random design, for which we prove optimality in the following sense: their convergence rate matches exactly or closely the ra…
General elliptic equations with spatially discontinuous diffusion coefficients may be used as a simplified model for subsurface flow in heterogeneous or fractured porous media. In such a model, data sparsity and measurement errors are often taken into account by a randomization of the diffusion coefficient of the elliptic equation which reveals the necessity of the construction of flexible, spatially discontinuous random fields. Subordinated Gaussian random fields are random functions on higher dimensional parameter domains with discontinuous sample paths and great distributional flexibility. …
Recently, so-called full-history recursive multilevel Picard (MLP) approximation schemes have been introduced and shown to overcome the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations (PDEs) with Lipschitz nonlinearities. The key contribution of this article is to introduce and analyze a new variant of MLP approximation schemes for certain semilinear elliptic PDEs with Lipschitz nonlinearities and to prove that the proposed approximation schemes overcome the curse of dimensionality in the numerical approximation of such semilinear e…
