In recent years, neural networks have achieved remarkable progress in various fields and have also drawn much attention in applying them on scientific problems. A line of methods involving neural networks for solving partial differential equations (PDEs), such as Physics-Informed Neural Networks (PINNs) and the Deep Ritz Method (DRM), has emerged. Although these methods outperform classical numerical methods in certain cases, the optimization problems involving neural networks are typically non-convex and non-smooth, which can result in unsatisfactory solutions for PDEs. In contrast to deterministic neural networks, the hidden weights of random neural networks are sampled from some prior distribution and only the output weights participate in training. This makes training much simpler, but it remains unclear how to select the prior distribution. In this paper, we focus on Barron type functions and approximate them under Sobolev norms by random neural networks with clear prior distribution. In addition to the approximation error, we also derive bounds for the optimization and generalization errors of random neural networks for solving PDEs when the solutions are Barron type functions.
In this paper, we develop and analyze a trilinear immersed finite element method for solving three-dimensional elliptic interface problems. The proposed method can be utilized on interface-unfitted meshes such as Cartesian grids consisting of cuboids. We establish the trace and inverse inequalities for trilinear IFE functions for interface elements with arbitrary interface-cutting configuration. Optimal a priori error estimates are rigorously proved in both energy and L2 norms. Numerical examples are provided not only to verify our theoretical results but also to demonstrate the applicability …
Least-Squares versus Partial Least-Squares Finite Element Methods: Robust A Priori and A Posteriori Error Estimates of Augmented Mixed Finite Element Methods
This paper analyzes the generalization error of two-layer neural networks for computing the ground state of the Schrödinger operator on a $d$-dimensional hypercube. We prove that the convergence rate of the generalization error is independent of the dimension $d$, under the a priori assumption that the ground state lies in a spectral Barron space. We verify such assumption by proving a new regularity estimate for the ground state in the spectral Barron space. The later is achieved by a fixed point argument based on the Krein-Rutman theorem.
In this paper, we propose a tensor type of discretization and optimization process for solving high dimensional partial differential equations. First, we design the tensor type of trial function for the high dimensional partial differential equations. Based on the tensor structure of the trial functions, we can do the direct numerical integration of the approximate solution without the help of Monte-Carlo method. Then combined with the Ritz or Galerkin method, solving the high dimensional partial differential equation can be transformed to solve a concerned optimization problem. Some numerical…
This article presents several numerical techniques for solving Laplace equation. A numerical FORTRAN solver is developed to solve the 2D laplace equation. The numerical approaches implemented in the solver include Jacobi, Gauss-Siedel, Successive Over Relaxation, Alternating Direct Implicit and Multigrid methods. Detailed comparison between different numerical methods is presented and discussed
Duke University（杜克大学）；University of Massachusetts（马萨诸塞大学）
This paper concerns the a priori generalization analysis of the Deep Ritz Method (DRM) [W. E and B. Yu, 2017], a popular neural-network-based method for solving high dimensional partial differential equations. We derive the generalization error bounds of two-layer neural networks in the framework of the DRM for solving two prototype elliptic PDEs: Poisson equation and static Schrödinger equation on the $d$-dimensional unit hypercube. Specifically, we prove that the convergence rates of generalization errors are independent of the dimension $d$, under the a priori assumption that the exact solu…
In this work, we derive a priori error estimate of the mixed residual method when solving some elliptic PDEs. Our work is the first theoretical study of this method. We prove that the neural network solutions will converge if we increase the training samples and network size without any constraint on the ratio of training samples to the network size. Besides, our results suggest that the mixed residual method can recover high order derivatives better than the deep Ritz method, which has also been verified by our numerical experiments.
A new penalty-free neural network method, PFNN-2, is presented for solving partial differential equations, which is a subsequent improvement of our previously proposed PFNN method [1]. PFNN-2 inherits all advantages of PFNN in handling the smoothness constraints and essential boundary conditions of self-adjoint problems with complex geometries, and extends the application to a broader range of non-self-adjoint time-dependent differential equations. In addition, PFNN-2 introduces an overlapping domain decomposition strategy to substantially improve the training efficiency without sacrificing ac…
Moving sample method for solving time-dependent partial differential equations
The Institute of Mathematical Sciences, ShanghaiTech University（上海科技大学数学科学学院）；Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院计算数学与科学工程计算研究所）；School of Mathematical Sciences, University of Chinese Academy of Sciences（中国科学院大学数学科学学院）；Department of Mathematics, School of Sciences, Great Bay University（大湾区大学理学院数学系）；Department of Mathematics, Center for Computation and Technology, Louisiana State University（路易斯安那州立大学计算与技术中心数学系）
A Priori Error Estimation of Physics-Informed Neural Networks Solving Allen--Cahn and Cahn--Hilliard Equations
SandGold AI Research（SandGold AI研究院）；University of Macau（澳门大学）；Xiamen University（厦门大学）；Sichuan University（四川大学）；South China Agricultural University（华南农业大学）；Macau University of Science and Technology（澳门科技大学）
We analyze neural network solutions to partial differential equations obtained with Physics Informed Neural Networks. In particular, we apply tools of classical finite element error analysis to obtain conclusions about the error of the Deep Ritz method applied to the Laplace and the Stokes equations. Further, we develop an a posteriori error estimator for neural network approximations of partial differential equations. The proposed approach is based on the dual weighted residual estimator. It is destined to serve as a stopping criterion that guarantees the accuracy of the solution independentl…
This paper presents a novel semi-analytical collocation method to solve multi-term variable-order time fractional partial differential equations (VOTFPDEs). In the proposed method it employs the Fourier series expansion for spatial discretization, which transforms the original multi-term VOTFPDEs into a sequence of multi-term variable-order time fractional ordinary differential equations (VOTFODEs). Then these VOTFODEs can be solved by using the recent-developed backward substitution method. Several numerical examples verify the accuracy and efficiency of the proposed numerical approach in the…
Physics Informed Neural Networks (PINNs) have recently gained popularity for solving partial differential equations, given the fact they escape the curse of dimensionality. In this paper, we present Physics Informed Neural Networks as an underdetermined point matching collocation method then expose the connection between Galerkin Least Square (GALS) and PINNs, to develop an a priori error estimate, in the context of elliptic problems. In particular, techniques that belong to the realm of least square finite elements and Rademacher complexity analysis are used to obtain the error estimate.
We present a novel method for using Neural Networks (NNs) for finding solutions to a class of Partial Differential Equations (PDEs). Our method builds on recent advances in Neural Radiance Field research (NeRFs) and allows for a NN to converge to a PDE solution much faster than classic Physically Informed Neural Network (PINNs) approaches.
An economic cascadic tensor multigrid method for solving high dimensional elliptic linear partial differential problems
Guilin University of Electronic Technology（桂林电子科技大学）；Guangxi Colleges and Universities Key Laboratory of Data Analysis and Computation（广西高校数据分析与计算重点实验室）；Center for Applied Mathematics of Guangxi (GUET)（广西应用数学中心）
At present, deep learning based methods are being employed to resolve the computational challenges of high-dimensional partial differential equations (PDEs). But the computation of the high order derivatives of neural networks is costly, and high order derivatives lack robustness for training purposes. We propose a novel approach to solving PDEs with high order derivatives by simultaneously approximating the function value and derivatives. We introduce intermediate variables to rewrite the PDEs into a system of low order differential equations as what is done in the local discontinuous Galerki…
Neural Networks have been widely used to solve Partial Differential Equations. These methods require to approximate definite integrals using quadrature rules. Here, we illustrate via 1D numerical examples the quadrature problems that may arise in these applications and propose different alternatives to overcome them, namely: Monte Carlo methods, adaptive integration, polynomial approximations of the Neural Network output, and the inclusion of regularization terms in the loss. We also discuss the advantages and limitations of each proposed alternative. We advocate the use of Monte Carlo methods…
Data-integrated neural networks for solving partial differential equations
Xiangtan University（湘潭大学）；National Center for Applied Mathematics in Hunan, Key Laboratory of Intelligent Computing & Information Processing of Ministry of Education, Xiangtan University（湖南应用数学中心，教育部智能计算与信息处理重点实验室，湘潭大学）；Hunan Key Laboratory for Computation and Simulation in Science and Engineering（湖南省科学与工程计算与仿真重点实验室；湘潭大学数学与计算科学学院）；School of Mathematics and Computational Science, Xiangtan University（贵州大学数学与统计学院）；School of Mathematics and Statistics, Guizhou University
Results and questions on a nonlinear approximation approach for solving high-dimensional partial differential equations
CERMICS, École des Ponts（巴黎路桥学院）；INRIA Rocquencourt（法国国家信息与自动化研究所罗克安库尔中心）；Laboratoire J.L.-Lions, Université Pierre et Marie Curie（皮埃尔和玛丽居里大学J.L.朗之万实验室）
In the present study, an interior-exterior penalty discontinuous Galerkin finite element method (DG-FEM) is analysed for solving Elastohydrodynamic lubrication (EHL) line and point contact problems. The existence of discrete penalized solution is examined using Brouwer's fixed point theorem. Furthermore, the uniqueness of solution is proved using Lipschitz continuity of the discrete solution map under light load parameter assumptions. A priori error estimates are achieved in $L^{2}$ and $H^{1}$ norms which are shown to be optimal in mesh size $h$ and suboptimal in polynomial degree $p$. The va…
Deep learning methods have achieved great success in solving partial differential equations (PDEs), where the loss is often defined as an integral. The accuracy and efficiency of these algorithms depend greatly on the quadrature method. We propose to apply quasi-Monte Carlo (QMC) methods to the Deep Ritz Method (DRM) for solving the Neumann problems for the Poisson equation and the static Schrödinger equation. For error estimation, we decompose the error of using the deep learning algorithm to solve PDEs into the generalization error, the approximation error and the training error. We establis…
Performing analog computations with metastructures is an emerging wave-based paradigm for solving mathematical problems. For such devices, one major challenge is their reconfigurability, especially without the need for a priori mathematical computations or computationally-intensive optimization. Their equation-solving capabilities are applied only to matrices with special spectral (eigenvalue) distribution. Here we report the theory and design of wave-based metastructures using tunable elements capable of solving integral/differential equations in a fully-reconfigurable fashion. We consider tw…
