We propose a novel composite framework to find unknown fields in the context of inverse problems for partial differential equations (PDEs). We blend the high expressibility of deep neural networks as universal function estimators with the accuracy and reliability of existing numerical algorithms for partial differential equations as custom layers in semantic autoencoders. Our design brings together techniques of computational mathematics, machine learning and pattern recognition under one umbrella to incorporate domain-specific knowledge and physical constraints to discover the underlying hidden fields. The network is explicitly aware of the governing physics through a hard-coded PDE solver layer in contrast to most existing methods that incorporate the governing equations in the loss function or rely on trainable convolutional layers to discover proper discretizations from data. This subsequently focuses the computational load to only the discovery of the hidden fields and therefore is more data efficient. We call this architecture Blended inverse-PDE networks (hereby dubbed BiPDE networks) and demonstrate its applicability for recovering the variable diffusion coefficient in Poisson problems in one and two spatial dimensions, as well as the diffusion coefficient in the time-dependent and nonlinear Burgers' equation in one dimension. We also show that this approach is robust to noise.
This paper studies an unsupervised deep learning-based numerical approach for solving partial differential equations (PDEs). The approach makes use of the deep neural network to approximate solutions of PDEs through the compositional construction and employs least-squares functionals as loss functions to determine parameters of the deep neural network. There are various least-squares functionals for a partial differential equation. This paper focuses on the so-called first-order system least-squares (FOSLS) functional studied in [3], which is based on a first-order system of scalar second-orde…
Shanghai Jiao Tong University（上海交通大学）；Shanghai Center for Brain Science and Brain-Inspired Technology（上海脑科学与类脑研究中心）；Peking University（北京大学）；AI for Science Institute（北京科学智能研究院）
In this paper, we propose a a machine learning approach via model-operator-data network (MOD-Net) for solving PDEs. A MOD-Net is driven by a model to solve PDEs based on operator representation with regularization from data. For linear PDEs, we use a DNN to parameterize the Green's function and obtain the neural operator to approximate the solution according to the Green's method. To train the DNN, the empirical risk consists of the mean squared loss with the least square formulation or the variational formulation of the governing equation and boundary conditions. For complicated problems, the…
Latent Neural Operator Pretraining for Solving Time-Dependent PDEs
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
There has been an arising trend of adopting deep learning methods to study partial differential equations (PDEs). In this paper, we introduce a deep recurrent framework for solving time-dependent PDEs without generating large scale data sets. We provide a new perspective, that is, a different type of architecture through exploring the possible connections between traditional numerical methods (such as finite difference schemes) and deep neural networks, particularly convolutional and fully-connected neural networks. Our proposed approach will show its effectiveness and efficiency in solving PD…
In this paper, we propose the idea of radial scaling in frequency domain and activation functions with compact support to produce a multi-scale DNN (MscaleDNN), which will have the multi-scale capability in approximating high frequency and high dimensional functions and speeding up the solution of high dimensional PDEs. Numerical results on high dimensional function fitting and solutions of high dimensional PDEs, using loss functions with either Ritz energy or least squared PDE residuals, have validated the increased power of multi-scale resolution and high frequency capturing of the proposed …
University of Notre Dame（圣母大学）；Duke Kunshan University（杜克昆山大学）
Partial differential equations (PDEs) play a crucial role in studying a vast number of problems in science and engineering. Numerically solving nonlinear and/or high-dimensional PDEs is often a challenging task. Inspired by the traditional finite difference and finite elements methods and emerging advancements in machine learning, we propose a sequence deep learning framework called Neural-PDE, which allows to automatically learn governing rules of any time-dependent PDE system from existing data by using a bidirectional LSTM encoder, and predict the next n time steps data. One critical featur…
DeltaPhi: Physical States Residual Learning for Neural Operators in Data-Limited PDE Solving
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The State Key Lab of Brain-Machine Intelligence, Zhejiang University（浙江大学脑机智能国家重点实验室）
BiLO: Bilevel Local Operator Learning for PDE Inverse Problems
Department of Mathematics, University of California, Irvine（加州大学伊文斯分校数学系）；Department of Computer Science, University of California, Irvine（加州大学伊文斯分校计算机科学系）；Department of Biomedical Engineering, University of California, Irvine（加州大学伊文斯分校生物医学工程系）
The Alternating Direction Method of Multipliers (ADMM) provides a natural way of solving inverse problems with multiple partial differential equations (PDE) forward models and nonsmooth regularization. ADMM allows splitting these large-scale inverse problems into smaller, simpler sub-problems, for which computationally efficient solvers are available. In particular, we apply large-scale second-order optimization methods to solve the fully-decoupled Tikhonov regularized inverse problems stemming from each PDE forward model. We use fast proximal methods to handle the nonsmooth regularization ter…
We propose a First-Order System Least Squares (FOSLS) method based on deep-learning for numerically solving second-order elliptic PDEs. The method we propose is capable of dealing with either variational and non-variational problems, and because of its meshless nature, it can also deal with problems posed in high-dimensional domains. We prove the $Γ$-convergence of the neural network approximation towards the solution of the continuous problem, and extend the convergence proof to some well-known related methods. Finally, we present several numerical examples illustrating the performance of our…
In this paper, we develop an asymptotic expansion-regularization (AER) method for inverse source problems in two-dimensional nonlinear and nonstationary singularly perturbed partial differential equations (PDEs). The key idea of this approach is the use of the asymptotic-expansion theory, which allows us to determine the conditions for the existence and uniqueness of a solution to a given PDE with a sharp transition layer. As a by-product, we derive a simpler link equation between the source function and first-order asymptotic approximation of the measurable quantities, and based on that equat…
While deep learning has achieved remarkable success in solving partial differential equations (PDEs), it still faces significant challenges, particularly when the PDE solutions have low regularity or singularities. To address these issues, we propose the Weak TransNet (WTN) method, based on a Petrov-Galerkin formulation, for solving elliptic PDEs in this work, though its framework may extend to other classes of equations. Specifically, the neural feature space defined by TransNet (Zhang et al., 2023) is used as the trial space, while the test space is composed of radial basis functions. Since …
We introduce a simple, rigorous, and unified framework for solving nonlinear partial differential equations (PDEs), and for solving inverse problems (IPs) involving the identification of parameters in PDEs, using the framework of Gaussian processes. The proposed approach: (1) provides a natural generalization of collocation kernel methods to nonlinear PDEs and IPs; (2) has guaranteed convergence for a very general class of PDEs, and comes equipped with a path to compute error bounds for specific PDE approximations; (3) inherits the state-of-the-art computational complexity of linear solvers fo…
Numerous physics theories are rooted in partial differential equations (PDEs). However, the increasingly intricate physics equations, especially those that lack analytic solutions or closed forms, have impeded the further development of physics. Computationally solving PDEs by classic numerical approaches suffers from the trade-off between accuracy and efficiency and is not applicable to the empirical data generated by unknown latent PDEs. To overcome this challenge, we present KoopmanLab, an efficient module of the Koopman neural operator family, for learning PDEs without analytic solutions o…
Physics-informed neural networks (PINNs) have been proposed to learn the solution of partial differential equations (PDE). In PINNs, the residual form of the PDE of interest and its boundary conditions are lumped into a composite objective function as soft penalties. Here, we show that this specific way of formulating the objective function is the source of severe limitations in the PINN approach when applied to different kinds of PDEs. To address these limitations, we propose a versatile framework based on a constrained optimization problem formulation, where we use the augmented Lagrangian m…
This paper introduces a new approximation scheme for solving high-dimensional semilinear partial differential equations (PDEs) and backward stochastic differential equations (BSDEs). First, we decompose a target semilinear PDE (BSDE) into two parts, namely "dominant" linear and "small" nonlinear PDEs. Then, we employ a Deep BSDE solver with a new control variate method to solve those PDEs, where approximations based on an asymptotic expansion technique are effectively applied to the linear part and also used as control variates for the nonlinear part. Moreover, our theoretical result indicates…
We investigate the iterative methods proposed by Maz'ya and Kozlov (see [3], [4]) for solving ill-posed reconstruction problems modeled by PDE's. We consider linear time dependent problems of elliptic, hyperbolic and parabolic types. Each iteration of the analyzed methods consists on the solution of a well posed boundary (or initial) value problem. The iterations are described as powers of affine operators, as in [4]. We give alternative convergence proofs for the algorithms, using spectral theory and some functional analytical results (see [5], [6]).
Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks
Tsinghua University（清华大学）；Bauhaus-Universität Weimar（魏玛包豪斯大学）；CNPC Engineering Technology RD Company Limited（中国石油集团工程技术研究院有限公司）；Queensland University of Technology（昆士兰科技大学）；Leibniz University Hannover（汉诺威莱布尼茨大学）
Northeastern University（东北大学）；University of Notre Dame（圣母大学）；MIT（麻省理工学院）
Partial differential equations (PDEs) play a fundamental role in modeling and simulating problems across a wide range of disciplines. Recent advances in deep learning have shown the great potential of physics-informed neural networks (PINNs) to solve PDEs as a basis for data-driven modeling and inverse analysis. However, the majority of existing PINN methods, based on fully-connected NNs, pose intrinsic limitations to low-dimensional spatiotemporal parameterizations. Moreover, since the initial/boundary conditions (I/BCs) are softly imposed via penalty, the solution quality heavily relies on h…
Freie Universität Berlin（柏林自由大学）；Zuse Institute Berlin（柏林祖斯研究所）；dida Datenschmiede GmbH（dida数据锻造有限责任公司）；University of Vienna（维也纳大学）
The combination of Monte Carlo methods and deep learning has recently led to efficient algorithms for solving partial differential equations (PDEs) in high dimensions. Related learning problems are often stated as variational formulations based on associated stochastic differential equations (SDEs), which allow the minimization of corresponding losses using gradient-based optimization methods. In respective numerical implementations it is therefore crucial to rely on adequate gradient estimators that exhibit low variance in order to reach convergence accurately and swiftly. In this article, we…
This paper develops meshless methods for probabilistically describing discretisation error in the numerical solution of partial differential equations. This construction enables the solution of Bayesian inverse problems while accounting for the impact of the discretisation of the forward problem. In particular, this drives statistical inferences to be more conservative in the presence of significant solver error. Theoretical results are presented describing rates of convergence for the posteriors in both the forward and inverse problems. This method is tested on a challenging inverse problem w…
We present a proof of concept for solving a 1+1D complex-valued, delay partial differential equation (PDE) that emerges in the study of waveguide quantum electrodynamics (QED) by adapting the finite-difference time-domain (FDTD) method. The delay term is spatially non-local, rendering conventional approaches such as the method of lines inapplicable. We show that by properly designing the grid and by supplying the (partial) exact solution as the boundary condition, the delay PDE can be numerically solved. In addition, we demonstrate that while the delay imposes strong data dependency, multi-thr…
Hessian operators arising in inverse problems governed by partial differential equations (PDEs) play a critical role in delivering efficient, dimension-independent convergence for both Newton solution of deterministic inverse problems, as well as Markov chain Monte Carlo sampling of posteriors in the Bayesian setting. These methods require the ability to repeatedly perform such operations on the Hessian as multiplication with arbitrary vectors, solving linear systems, inversion, and (inverse) square root. Unfortunately, the Hessian is a (formally) dense, implicitly-defined operator that is int…
The intrusive (sample-free) spectral stochastic finite element method (SSFEM) is a powerful numerical tool for solving stochastic partial differential equations (PDEs). However, it is not widely adopted in academic and industrial applications because it demands intrusive adjustments in the PDE solver, which require substantial coding efforts compared to the non-intrusive (sampling) SSFEM. Using an example of stochastic PDE, in this article, we demonstrate that the implementational challenges of the intrusive approach can be alleviated using FEniCS -- a general purpose finite element package an…
This paper studies numerical solutions for parameterized partial differential equations (P-PDEs) with deep learning (DL). P-PDEs arise in many important application areas and the computational cost using traditional numerical schemes can be exorbitant, especially when the parameters fall into a particular range and the underlying PDE is required to be solved with high accuracy. Recently, solving PDEs with DL has become an emerging field. Existing works demonstrate great potentials of the DL based approach in speeding up numerical solutions of PDEs. However, there is still limited research on t…
Learning-guided Kansa collocation for forward and inverse PDEs beyond linearity
Department of Computer Science and Technology（计算机科学与技术系）；Department of Applied Mathematics and Theoretical Physics（应用数学与理论物理系）；University of Cambridge（剑桥大学）
