Parametric PDEs: Sparse or Low-Rank Approximations?
Hausdorff Center for Mathematics（豪斯多夫数学中心）；Institute for Numerical Simulation（数值模拟研究所）；Sorbonne Universités, UPMC Univ Paris 06, CNRS, UMR 7598, Laboratoire Jacques-Louis Lions（索邦大学，巴黎第六大学，法国国家科学研究中心，UMR 7598，雅克-路易·利翁斯实验室）；Institut für Geometrie und Praktische Mathematik, RWTH Aachen（亚琛工业大学几何与应用数学研究所）
We consider adaptive approximations of the parameter-to-solution map for elliptic operator equations depending on a large or infinite number of parameters, comparing approximation strategies of different degrees of nonlinearity: sparse polynomial expansions, general low-rank approximations separating spatial and parametric variables, and hierarchical tensor decompositions separating all variables. We describe corresponding adaptive algorithms based on a common generic template and show their near-optimality with respect to natural approximability assumptions for each type of approximation. A central ingredient in the resulting bounds for the total computational complexity are new operator compression results for the case of infinitely many parameters. We conclude with a comparison of the complexity estimates based on the actual approximability properties of classes of parametric model problems, which shows that the computational costs of optimized low-rank expansions can be significantly lower or higher than those of sparse polynomial expansions, depending on the particular type of parametric problem.
We consider a linear elliptic partial differential equation (PDE) with a generic uniformly bounded parametric coefficient. The solution to this PDE problem is approximated in the framework of stochastic Galerkin finite element methods. We perform a posteriori error analysis of Galerkin approximations and derive a reliable and efficient estimate for the energy error in these approximations. Practical versions of this error estimate are discussed and tested numerically for a model problem with non-affine parametric representation of the coefficient. Furthermore, we use the error reduction indica…
In this paper we introduce an algorithm based on a sparse grid adaptive refinement, for the approximation of the eigensolutions to parametric problems arising from elliptic partial differential equations. In particular, we are interested in detecting the crossing of the hypersurfaces describing the eigenvalues as a function of the parameters. The a priori matching is followed by an a posteriori verification, driven by a suitably defined error indicator. At a given refinement level, a sparse grid approach is adopted for the construction of the grid of the next level, by using the marking given …
We investigate non-adaptive methods of deep ReLU neural network approximation in Bochner spaces $L_2({\mathbb U}^\infty, X, μ)$ of functions on ${\mathbb U}^\infty$ taking values in a separable Hilbert space $X$, where ${\mathbb U}^\infty$ is either ${\mathbb R}^\infty$ equipped with the standard Gaussian probability measure, or ${\mathbb I}^\infty:= [-1,1]^\infty$ equipped with the Jacobi probability measure. Functions to be approximated are assumed to satisfy a certain weighted $\ell_2$-summability of the generalized chaos polynomial expansion coefficients with respect to the measure $μ$. We…
Sampling recovery in Bochner spaces and applications to parametric PDEs
Mathematisch-Geographische Fakultät, KU Eichstätt-Ingolstadt（凯泽大学艾希施泰特-因戈尔施塔特数学地理学院）；Information Technology Institute, Vietnam National University, Hanoi（越南国家大学河内信息技术研究所）
We obtained convergence rates of the collocation approximation by deep ReLU neural networks of solutions to elliptic PDEs with lognormal inputs, parametrized by $\boldsymbol{y}$ from the non-compact set $\mathbb{R}^\infty$. The approximation error is measured in the norm of the Bochner space $L_2(\mathbb{R}^\infty, V, γ)$, where $γ$ is the infinite tensor product standard Gaussian probability measure on $\mathbb{R}^\infty$ and $V$ is the energy space. We also obtained similar results for the case when the lognormal inputs are parametrized on $\mathbb{R}^M$ with very large dimension $M$, and th…
Iowa State University（爱荷华州立大学）；New York University（纽约大学）
We consider a mesh-based approach for training a neural network to produce field predictions of solutions to parametric partial differential equations (PDEs). This approach contrasts current approaches for "neural PDE solvers" that employ collocation-based methods to make point-wise predictions of solutions to PDEs. This approach has the advantage of naturally enforcing different boundary conditions as well as ease of invoking well-developed PDE theory -- including analysis of numerical stability and convergence -- to obtain capacity bounds for our proposed neural networks in discretized domai…
In this paper we discuss a projection model order reduction (MOR) method for a class of parametric linear evolution PDEs, which is based on the application of the Laplace transform. The main advantage of this approach consists in the fact that, differently from time stepping methods, like Runge-Kutta integrators, the Laplace transform allows to compute the solution directly at a given instant, which can be done by approximating the contour integral associated to the inverse Laplace transform by a suitable quadrature formula. In terms of some classical MOR methodology, this determines a signifi…
Sparse polynomial approximation for optimal control problems constrained by elliptic PDEs with lognormal random coefficients
Oden Institute for Computational Engineering & Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校奥登计算工程与科学学院）；Department of Mechanical Engineering, and Department of Geological Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校机械工程系和地质科学系）
This work is a follow-up to our previous contribution ("Convergence of sparse collocation for functions of countably many Gaussian random variables (with application to elliptic PDEs)", SIAM J. Numer. Anal., 2018), and contains further insights on some aspects of the solution of elliptic PDEs with lognormal diffusion coefficients using sparse grids. Specifically, we first focus on the choice of univariate interpolation rules, advocating the use of Gaussian Leja points as introduced by Narayan and Jakeman ("Adaptive Leja sparse grid constructions for stochastic collocation and high-dimensional …
The weighted essentially non-oscillatory (WENO) schemes are a popular class of high order accurate numerical methods for solving hyperbolic partial differential equations (PDEs). However when the spatial dimensions are high, the number of spatial grid points increases significantly. It leads to large amount of operations and computational costs in the numerical simulations by using nonlinear high order accuracy WENO schemes such as a fifth order WENO scheme. How to achieve fast simulations by high order WENO methods for high spatial dimension hyperbolic PDEs is a challenging and important ques…
Technische Universität Berlin（柏林工业大学）；University of Tromsø（特罗姆瑟大学）；University of Vienna（维也纳大学）
We derive upper bounds on the complexity of ReLU neural networks approximating the solution maps of parametric partial differential equations. In particular, without any knowledge of its concrete shape, we use the inherent low-dimensionality of the solution manifold to obtain approximation rates which are significantly superior to those provided by classical neural network approximation results. Concretely, we use the existence of a small reduced basis to construct, for a large variety of parametric partial differential equations, neural networks that yield approximations of the parametric sol…
Local Gevrey regularity and Quasi-Monte Carlo quadrature for PDEs parameterized on non-compact domains
Vietnam National Foundation for Science and Technology Development (NAFOSTED)（越南国家科学技术基金会）；Vietnam Institute for Advanced Study in Mathematics (VIASM)（越南高等数学研究所）；Hausdorff Research Institute for Mathematics（豪斯多夫数学研究所）；University of Oldenburg（奥尔登堡大学）
When working with PDEs the reconstruction of a previous state often proves difficult. Good prior knowledge and fast computational methods are crucial to build a working reconstruction. We want to identify the heat sources on a three dimensional domain from later measurements under the assumption of small, distinct sources, such as hot chippings from a milling tool. This leads us to the need for a Prior reflecting this a priori information. Sparsity-inducing hyperpriors have proven useful for similar problems with sparse signal or image reconstruction. We combine the method of using a hierarchi…
We develop the uniform sparse Fast Fourier Transform (usFFT), an efficient, non-intrusive, adaptive algorithm for the solution of elliptic partial differential equations with random coefficients. The algorithm is an adaption of the sparse Fast Fourier Transform (sFFT), a dimension-incremental algorithm, which tries to detect the most important frequencies in a given search domain and therefore adaptively generates a suitable Fourier basis corresponding to the approximately largest Fourier coefficients of the function. The usFFT does this w.r.t. the stochastic domain of the PDE simultaneously f…
University of Tübingen（图宾根大学）；University of Genova（热那亚大学）；Istituto Italiano di Tecnologia（意大利技术研究院）；Massachusetts Institute of Technology（麻省理工学院）；Max-Planck-Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Neural operators are a type of deep architecture that learns to solve (i.e. learns the nonlinear solution operator of) partial differential equations (PDEs). The current state of the art for these models does not provide explicit uncertainty quantification. This is arguably even more of a problem for this kind of tasks than elsewhere in machine learning, because the dynamical systems typically described by PDEs often exhibit subtle, multiscale structure that makes errors hard to spot by humans. In this work, we first provide a mathematically detailed Bayesian formulation of the ''shallow'' (li…
In this paper, we propose a network model, the multiclass classification-based reduced order model (MC-ROM), for solving time-dependent parametric partial differential equations (PPDEs). This work is inspired by the observation of applying the deep learning-based reduced order model (DL-ROM) to solve diffusion-dominant PPDEs. We find that the DL-ROM has a good approximation for some special model parameters, but it cannot approximate the drastic changes of the solution as time evolves. Based on this fact, we classify the dataset according to the magnitude of the solutions and construct corresp…
Recent experiments have shown that deep networks can approximate solutions to high-dimensional PDEs, seemingly escaping the curse of dimensionality. However, questions regarding the theoretical basis for such approximations, including the required network size, remain open. In this paper, we investigate the representational power of neural networks for approximating solutions to linear elliptic PDEs with Dirichlet boundary conditions. We prove that when a PDE's coefficients are representable by small neural networks, the parameters required to approximate its solution scale polynomially with t…
The Deep Operator Networks~(DeepONet) is a fundamentally different class of neural networks that we train to approximate nonlinear operators, including the solution operator of parametric partial differential equations (PDE). DeepONets have shown remarkable approximation and generalization capabilities even when trained with relatively small datasets. However, the performance of DeepONets deteriorates when the training data is polluted with noise, a scenario that occurs very often in practice. To enable DeepONets training with noisy data, we propose using the Bayesian framework of replica-exch…
Multi-index Stochastic Collocation convergence rates for random PDEs with parametric regularity
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Università di Pavia（帕维亚大学）；CNR-IMATI（意大利国家研究委员会应用数学研究所）
Partial Differential Equations (PDEs) describe several problems relevant to many fields of applied sciences, and their discrete counterparts typically involve the solution of sparse linear systems. In this context, we focus on the analysis of the computational aspects related to the solution of large and sparse linear systems with HPC solvers, by considering the performances of direct and iterative solvers in terms of computational efficiency, scalability, and numerical accuracy. Our aim is to identify the main criteria to support application-domain specialists in the selection of the most sui…
