We provide the details of an implementation of Fourier techniques for solving second-order linear partial differential equations (with constant coefficients) using a computer algebra system. The general Sturm-Liouville problem for the heat, wave and Laplace operators on the most common bounded domains is covered, as well as the general second-order linear parabolic equation with constant coefficients, which includes cases such as the convection-diffusion equation, by reduction to the heat equation.
Based on the Fourier extension, we propose an oversampling collocation method for solving the elliptic partial differential equations with variable coefficients over arbitrary irregular domains. This method only uses the function values on the equispaced nodes, which has low computational cost and versatility. While a variety of numerical experiments are presented to demonstrate the effectiveness of this method, it shows that the approximation error fast reaches a plateau with increasing the degrees of freedom, due to the inherent ill-conditioned of frames.
When using Neural Networks as trial functions to numerically solve PDEs, a key choice to be made is the loss function to be minimised, which should ideally correspond to a norm of the error. In multiple problems, this error norm coincides with--or is equivalent to--the $H^{-1}$-norm of the residual; however, it is often difficult to accurately compute it. This work assumes rectangular domains and proposes the use of a Discrete Sine/Cosine Transform to accurately and efficiently compute the $H^{-1}$ norm. The resulting Deep Fourier-based Residual (DFR) method efficiently and accurately approxim…
We develop the uniform sparse Fast Fourier Transform (usFFT), an efficient, non-intrusive, adaptive algorithm for the solution of elliptic partial differential equations with random coefficients. The algorithm is an adaption of the sparse Fast Fourier Transform (sFFT), a dimension-incremental algorithm, which tries to detect the most important frequencies in a given search domain and therefore adaptively generates a suitable Fourier basis corresponding to the approximately largest Fourier coefficients of the function. The usFFT does this w.r.t. the stochastic domain of the PDE simultaneously f…
In his monograph Chebyshev and Fourier Spectral Methods, John Boyd claimed that, regarding Fourier spectral methods for solving differential equations, ``[t]he virtues of the Fast Fourier Transform will continue to improve as the relentless march to larger and larger [bandwidths] continues''. This paper attempts to further the virtue of the Fast Fourier Transform (FFT) as not only bandwidth is pushed to its limits, but also the dimension of the problem. Instead of using the traditional FFT however, we make a key substitution: a high-dimensional, sparse Fourier transform (SFT) paired with rando…
Enhancing Solutions for Complex PDEs: Introducing Complementary Convolution and Equivariant Attention in Fourier Neural Operators
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Fourier neural operators (FNOs) have recently been proposed as an effective framework for learning operators that map between infinite-dimensional spaces. We prove that FNOs are universal, in the sense that they can approximate any continuous operator to desired accuracy. Moreover, we suggest a mechanism by which FNOs can approximate operators associated with PDEs efficiently. Explicit error bounds are derived to show that the size of the FNO, approximating operators associated with a Darcy type elliptic PDE and with the incompressible Navier-Stokes equations of fluid dynamics, only increases …
Multigrid methods are popular for solving linear systems derived from discretizing PDEs. Local Fourier Analysis (LFA) is a technique for investigating and tuning multigrid methods. P-multigrid is popular for high-order or spectral finite element methods, especially on unstructured meshes. In this paper, we introduce LFAToolkit.jl, a new Julia package for LFA of high-order finite element methods. LFAToolkit.jl analyzes preconditioning techniques for arbitrary systems of second order PDEs and supports mixed finite element methods. Specifically, we develop LFA of p-multigrid with arbitrary second…
Fourier continuation is an approach used to create periodic extensions of non-periodic functions in order to obtain highly-accurate Fourier expansions. These methods have been used in PDE-solvers and have demonstrated high-order convergence and spectrally accurate dispersion relations in numerical experiments. Discontinuous Galerkin (DG) methods are increasingly used for solving PDEs and, as all Galerkin formulations, come with a strong framework for proving stability and convergence. Here we propose the use of Fourier continuation in forming a new basis for the DG framework.
In this paper, we employed linear algebra and functional analysis to determine necessary and sufficient conditions for oscillation-free and stable solutions to linear and nonlinear parabolic partial differential equations. We applied singular value decomposition and Fourier analysis to various finite difference schemes to extract patterns in the eigenfunctions (sampled by the eigenvectors) and the shape of their eigenspectrum. With these, we determined how the initial and boundary conditions affect the frequency and long term behavior of numerical oscillations, as well as the location of solut…
The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an entire family of PDEs, in contrast to classical methods which solve one instance of the equation. In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fou…
California Institute of Technology（加州理工学院）；ETH Zurich（苏黎世联邦理工学院）
A large class of hyperbolic and advection-dominated PDEs can have solutions with discontinuities. This paper investigates, both theoretically and empirically, the operator learning of PDEs with discontinuous solutions. We rigorously prove, in terms of lower approximation bounds, that methods which entail a linear reconstruction step (e.g. DeepONet or PCA-Net) fail to efficiently approximate the solution operator of such PDEs. In contrast, we show that certain methods employing a non-linear reconstruction mechanism can overcome these fundamental lower bounds and approximate the underlying opera…
A Spectral-Based Method for Network-Formation PDEs
University of Arizona（亚利桑那大学）；Institut de Mathématiques de Toulouse, CNRS & Université de Toulouse（图卢兹数学研究所，法国国家科学研究中心与图卢兹大学）；Faculty of Mathematics, University of Vienna（维也纳大学数学系）
In this paper we introduce a new approach to compute rigorously solutions of Cauchy problems for a class of semi-linear parabolic partial differential equations. Expanding solutions with Chebyshev series in time and Fourier series in space, we introduce a zero finding problem $F(a)=0$ on a Banach algebra $X$ of Fourier-Chebyshev sequences, whose solution solves the Cauchy problem. The challenge lies in the fact that the linear part $\mathcal{L} = DF(0)$ has an infinite block diagonal structure with blocks becoming less and less diagonal dominant at infinity. We introduce analytic estimates to …
To solve linear PDEs on metric graphs with standard coupling conditions (continuity and Kirchhoff's law), we develop and compare a spectral, a second-order finite difference, and a discontinuous Galerkin method. The spectral method yields eigenvalues and eigenvectors of arbitary order with machine precision and converges exponentially. These eigenvectors provide a Fourier-like basis on which to expand the solution; however, more complex coupling conditions require additional research. The discontinuous Galerkin method provides approximations of arbitrary polynomial order; however computing hig…
Robust physics (e.g., governing equations and laws) discovery is of great interest for many engineering fields and explainable machine learning. A critical challenge compared with general training is that the term and format of governing equations is not known as a prior. In addition, significant measurement noise and complex algorithm hyperparameter tuning usually reduces the robustness of existing methods. A robust data-driven method is proposed in this study for identifying the governing Partial Differential Equations (PDEs) of a given system from noisy data. The proposed method is based on…
Sparse Identification of Nonlinear Dynamics (SINDy) is a method of system discovery that has been shown to successfully recover governing dynamical systems from data (Brunton et al., PNAS, '16; Rudy et al., Sci. Adv. '17). Recently, several groups have independently discovered that the weak formulation provides orders of magnitude better robustness to noise. Here we extend our Weak SINDy (WSINDy) framework introduced in (arXiv:2005.04339) to the setting of partial differential equations (PDEs). The elimination of pointwise derivative approximations via the weak form enables effective machine-p…
Rigorous numerics for ill-posed PDEs: periodic orbits in the Boussinesq equation
VU University Amsterdam（阿姆斯特丹自由大学）；Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo（圣保罗大学数学与计算科学学院）；Université Laval, Département de Mathématiques et de Statistique（拉瓦尔大学数学与统计系）
