We present a novel linear solver that works well for large systems obtained from discretizing PDEs. It is robust and, for the examples we studied, the computational effort scales linearly with the number of equations. The algorithm is based on a wavelength decomposition that combines conjugate gradient, multi-scaling and iterative splitting methods into a single approach. On the surface, the algorithm is a simple preconditioned conjugate gradient with all the sophistication of the algorithm in the choice of the preconditioning matrix. The preconditioner is a very good approximate inverse of the linear operator. It is constructed from the inverse of the coarse grained linear operator and from smoothing operators that are based on an operator splitting on the fine grid. The coarse graining captures the long wavelength behavior of the inverse operator while the smoothing operator captures the short wavelength behavior. The conjugate gradient iteration accounts for the coupling between long and short wavelengths. The coarse grained operator corresponds to a lowerresolution approximation to the PDEs. While the coarse grained inverse is not known explicitly, the algorithm only requires that the preconditioner can be a applied to a vector. The coarse inverse applied to a vector can be obtained as the solution of another preconditioned conjugate gradient solver that applies the same algorithm to the smaller problem. Thus, the method is naturally recursive. The recursion ends when the matrix is sufficiently small for a solution to be obtained efficiently with a standard solver. We have tested our solver on the porous flow equation. On a workstation we have solved problems on grids ranging in dimension from $10^3$ to $10^6$, and found that the linear scaling holds.
This paper introduces a new approximation scheme for solving high-dimensional semilinear partial differential equations (PDEs) and backward stochastic differential equations (BSDEs). First, we decompose a target semilinear PDE (BSDE) into two parts, namely "dominant" linear and "small" nonlinear PDEs. Then, we employ a Deep BSDE solver with a new control variate method to solve those PDEs, where approximations based on an asymptotic expansion technique are effectively applied to the linear part and also used as control variates for the nonlinear part. Moreover, our theoretical result indicates…
Fast and accurate solution of time-dependent partial differential equations (PDEs) is of key interest in many research fields including physics, engineering, and biology. Generally, implicit schemes are preferred over the explicit ones for better stability and correctness. The existing implicit schemes are usually iterative and employ a general-purpose solver which may be sub-optimal for a specific class of PDEs. In this paper, we propose a neural solver to learn an optimal iterative scheme for a class of PDEs, in a data-driven fashion. We attain this objective by modifying an iteration of an …
The DD-CPM software library provides a set of tools for the discretization and solution of problems arising from the closest point method (CPM) for partial differential equations on surfaces. The solvers are built on top of the well-known PETSc framework, and are supplemented by custom domain decomposition (DD) preconditioners specific to the CPM. These solvers are fully compatible with distributed memory parallelism through MPI. This library is particularly well suited to the solution of elliptic and parabolic equations, including many reaction-diffusion equations. The software is detailed he…
We develop a boundary integral equation solver for elliptic partial differential equations on complex \threed geometries. Our method is efficient, high-order accurate and robustly handles complex geometries. A key component is our singular and near-singular layer potential evaluation scheme, \qbkix: a simple extrapolation of the solution along a line to the boundary. We present a series of geometry-processing algorithms required for \qbkix to run efficiently with accuracy guarantees on arbitrary geometries and an adaptive upsampling scheme based on a iteration-free heuristic for quadrature err…
In this paper we introduce a procedure for identifying optimal methods in parametric families of numerical schemes for initial value problems in partial differential equations. The procedure maximizes accuracy by adaptively computing optimal parameters that minimize a defect-based estimate of the local error at each time-step. Viable refinements are proposed to reduce the computational overheads involved in the solution of the optimization problem, and to maintain conservation properties of the original methods. We apply the new strategy to recently introduced families of conservative schemes …
Partial Differential Equations (PDEs) describe several problems relevant to many fields of applied sciences, and their discrete counterparts typically involve the solution of sparse linear systems. In this context, we focus on the analysis of the computational aspects related to the solution of large and sparse linear systems with HPC solvers, by considering the performances of direct and iterative solvers in terms of computational efficiency, scalability, and numerical accuracy. Our aim is to identify the main criteria to support application-domain specialists in the selection of the most sui…
The paper introduces a very simple and fast computation method for high-dimensional integrals to solve high-dimensional Kolmogorov partial differential equations (PDEs). The new machine learning-based method is obtained by solving a stochastic weighted minimization with stochastic gradient descent which is inspired by a high-order weak approximation scheme for stochastic differential equations (SDEs) with Malliavin weights. Then solutions to high-dimensional Kolmogorov PDEs or expectations of functionals of solutions to high-dimensional SDEs are accurately approximated without suffering from t…
How to build an accurate reduced order model (ROM) for multidimensional time dependent partial differential equations (PDEs) is quite open. In this paper, we propose a new ROM for linear parabolic PDEs. We prove that our new method can be orders of magnitude faster than standard solvers, and is also much less memory intensive. Under some assumptions on the problem data, we prove that the convergence rates of the new method is the same with standard solvers. Numerical experiments are presented to confirm our theoretical result.
Weizmann Institute of Science（魏茨曼科学研究所）；Technion, Israel Institute of Technology（以色列理工学院）
Constructing fast numerical solvers for partial differential equations (PDEs) is crucial for many scientific disciplines. A leading technique for solving large-scale PDEs is using multigrid methods. At the core of a multigrid solver is the prolongation matrix, which relates between different scales of the problem. This matrix is strongly problem-dependent, and its optimal construction is critical to the efficiency of the solver. In practice, however, devising multigrid algorithms for new problems often poses formidable challenges. In this paper we propose a framework for learning multigrid sol…
Technical University Munich（慕尼黑工业大学）；École normale supérieure, Paris（巴黎高等师范学院）；deepc GmbH, Munich（慕尼黑deepc有限公司）；Helmholtz Zentrum München（慕尼黑亥姆霍兹研究中心）；University of Zurich（苏黎世大学）
Fast and accurate solutions of time-dependent partial differential equations (PDEs) are of pivotal interest to many research fields, including physics, engineering, and biology. Generally, implicit/semi-implicit schemes are preferred over explicit ones to improve stability and correctness. However, existing semi-implicit methods are usually iterative and employ a general-purpose solver, which may be sub-optimal for a specific class of PDEs. In this paper, we propose a neural solver to learn an optimal iterative scheme in a data-driven fashion for any class of PDEs. Specifically, we modify a si…
We introduce a fast direct solver for variable-coefficient elliptic partial differential equations on surfaces based on the hierarchical Poincaré-Steklov method. The method takes as input an unstructured, high-order quadrilateral mesh of a surface and discretizes surface differential operators on each element using a high-order spectral collocation scheme. Elemental solution operators and Dirichlet-to-Neumann maps tangent to the surface are precomputed and merged in a pairwise fashion to yield a hierarchy of solution operators that may be applied in $\mathcal{O}(N \log N)$ operations for a mes…
In this article, we derive a new, fast, and robust preconditioned iterative solution strategy for the all-at-once solution of optimal control problems with time-dependent PDEs as constraints, including the heat equation and the non-steady convection--diffusion equation. After applying an optimize-then-discretize approach, one is faced with continuous first-order optimality conditions consisting of a coupled system of PDEs. As opposed to most work in preconditioning the resulting discretized systems, where a (first-order accurate) backward Euler method is used for the discretization of the time…
UGrid: An Efficient-And-Rigorous Neural Multigrid Solver for Linear PDEs
Stony Brook University (SUNY)（纽约州立大学石溪分校）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）
This paper studies numerical solutions for parameterized partial differential equations (P-PDEs) with deep learning (DL). P-PDEs arise in many important application areas and the computational cost using traditional numerical schemes can be exorbitant, especially when the parameters fall into a particular range and the underlying PDE is required to be solved with high accuracy. Recently, solving PDEs with DL has become an emerging field. Existing works demonstrate great potentials of the DL based approach in speeding up numerical solutions of PDEs. However, there is still limited research on t…
Memory bound applications such as solvers for large sparse systems of equations remain a challenge for GPUs. Fast solvers should be based on numerically efficient algorithms and implemented such that global memory access is minimised. To solve systems with up to one trillion ($10^{12}$) unknowns the code has to make efficient use of several million individual processor cores on large GPU clusters. We describe the multi-GPU implementation of two algorithmically optimal iterative solvers for anisotropic elliptic PDEs which are encountered in atmospheric modelling. In this application the conditi…
IGA-based Multi-Index Stochastic Collocation for random PDEs on arbitrary domains
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；Consiglio Nazionale delle Ricerche - Istituto di Matematica Applicata e Tecnologie Informatiche “E. Magenes” (CNR-IMATI)（意大利国家研究委员会应用数学与信息技术研究所）；RWTH Aachen University（亚琛工业大学）
We prove quantitative norm bounds for a family of operators involving impedance boundary conditions on convex, polygonal domains. A robust numerical construction of Helmholtz scattering solutions in variable media via the Dirichlet-to-Neumann operator involves a decomposition of the domain into a sequence of rectangles of varying scales and constructing impedance-to-impedance boundary operators on each subdomain. Our estimates in particular ensure the invertibility, with quantitative bounds in the frequency, of the merge operators required to reconstruct the original Dirichlet-to-Neumann opera…
Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Space
The State Key Lab of Brain-Machine Intelligence, Zhejiang University, Hangzhou, China（脑机智能国家重点实验室，浙江大学，杭州，中国）；College of Computer Science and Technology, Zhejiang University, Hangzhou, China（计算机科学与技术学院，浙江大学，杭州，中国）
A fast direct solver based neural network for solving PDEs
Department of Computer Science & Engineering, International Institute of Information Technology Bangalore (IIIT-B), India（计算机科学与工程系，国际信息学院班加罗尔（IIIT-B），印度）；Department of Data Science and Artificial Intelligence, International Institute of Information Technology Bangalore (IIIT-B), India（数据科学与人工智能系，国际信息学院班加罗尔（IIIT-B），印度）
Deep Truncated FBSDE Method: A Robust Solver for High-Dimensional Nonlinear PDEs and Fully Coupled FBSDEs
Research Institute of Intelligent Complex Systems, Fudan University（复旦大学智能复杂系统研究院）；Department of Mathematics, Friedrich-Alexander-Universität Erlangen–Nürnberg（埃尔朗根-纽伦堡大学数学系）
Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences (CAS)（中国科学院数学与系统科学研究院）；Beijing Institute of Technology（北京理工大学）；Zhongguancun Academy（中关村学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
The Alan Turing Institute（艾伦·图灵研究所）；University of Edinburgh（爱丁堡大学）；Vega
We develop several deep learning algorithms for approximating families of parametric PDE solutions. The proposed algorithms approximate solutions together with their gradients, which in the context of mathematical finance means that the derivative prices and hedging strategies are computed simulatenously. Having approximated the gradient of the solution one can combine it with a Monte-Carlo simulation to remove the bias in the deep network approximation of the PDE solution (derivative price). This is achieved by leveraging the Martingale Representation Theorem and combining the Monte Carlo sim…
The multigrid algorithm is an efficient numerical method for solving a variety of elliptic partial differential equations (PDEs). The method damps errors at progressively finer grid scales, resulting in faster convergence compared to standard iterative methods such as Gauss-Seidel. The prolongation, or coarse-to-fine interpolation operator within the multigrid algorithm lends itself to a data-driven treatment with ML super resolution, commonly used to increase the resolution of images. We (i) propose the novel integration of a super resolution generative adversarial network (GAN) model with th…
Optimal solvers for fourth-order PDEs discretized on unstructured grids
LSEC, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and System Sciences, Chinese Academy of Sciences（中国科学院数学与系统科学研究院计算数学与科学工程计算研究所）；Center for Computational Mathematics and Applications, Department of Mathematics, The Pennsylvania State University（宾夕法尼亚州立大学数学系计算数学与应用中心）
The discretization of surface intrinsic elliptic partial differential equations (PDEs) poses interesting challenges not seen in flat space. The discretization of these PDEs typically proceeds by either parametrizing the surface, triangulating the surface, or embedding the surface in a higher dimensional flat space. The closest point method (CPM) is an embedding method that represents surfaces using a function that maps points in the embedding space to their closest points on the surface. In the CPM, this mapping also serves as an extension operator that brings surface intrinsic data onto the e…
We consider parametric families of partial differential equations--PDEs where the parameter $κ$ modifies only the (1,1) block of a saddle point matrix product of a discretization below. The main goal is to develop an algorithm that removes, as much as possible, the dependence of iterative solvers on the parameter $κ$. The algorithm we propose requires only one matrix factorization which does not depend on $κ$, therefore, allows to reuse it for solving very fast a large number of discrete PDEs for different $κ$ and forcing terms. The design of the proposed algorithm is motivated by previous wor…
