The purpose of this work is to introduce a new idea of how to avoid the factorization of large matrices during the solution of stiff systems of ODEs. Starting from the general form of an explicit linear multistep method we suggest to adaptively choose its coefficients on each integration step in order to minimize the norm of the residual of an implicit BDF formula. Thereby we reduce the number of unknowns on each step from $n$ to $O(1)$, where $n$ is the dimension of the ODE system. We call this type of methods Minimal Residual Multistep (MRMS) methods. In the case of linear non-autonomous problem, besides the evaluations of the right-hand side of ODE, the resulting numerical scheme additionally requires one solution of a linear least-squares problem with a thin matrix per step. We show that the order of the method and its zero-stability properties coincide with those of the used underlying BDF formula. For the simplest analog of the implicit Euler method the properties of linear stability are investigated. Though the classical absolute stability analysis is not fully relevant to the MRMS methods, it is shown that this one-step method is applicable in stiff case. In the numerical experiment section we consider the fixed-step integration of a two-dimensional non-autonomous heat equation using the MRMS methods and their classical BDF counterparts. The starting values are taken from a preset slowly-varying exact solution. The comparison showed that both methods give similar numerical solutions, but in the case of large systems the MRMS methods are faster, and their advantage considerably increases with the growth of dimension. Python code with the experimantal code can be downloaded from the GitHub repository https://github.com/bfaleichik/mrms.
Many hyperbolic and kinetic equations contain a non-stiff convection/transport part and a stiff relaxation/collision part (characterized by the relaxation or mean free time $\varepsilon$). To solve this type of problems, implicit-explicit (IMEX) multistep methods have been widely used and their performance is understood well in the non-stiff regime ($\varepsilon=O(1)$) and limiting regime ($\varepsilon\rightarrow 0$). However, in the intermediate regime (say, $\varepsilon=O(Δt)$), uniform accuracy has been reported numerically without a complete theoretical justification (except some asymptoti…
To address the issues of stability and accuracy for reaction-diffusion equations, the development of high order and stable time-stepping methods is necessary. This is particularly true in the context of cardiac electrophysiology, where reaction-diffusion equations are coupled with stiff ODE systems. Many research have been led in that way in the past 15 years concerning implicit-explicit methods and exponential integrators. In 2009, Perego and Veneziani proposed an innovative time-stepping method of order 2. In this paper we present the extension of this method to the orders 3 and 4 and introd…
Department of Applied Physics and Applied Mathematics, Columbia University（哥伦比亚大学应用物理与应用数学系）
Linear multistep methods (LMMs) are popular time discretization techniques for the numerical solution of differential equations. Traditionally they are applied to solve for the state given the dynamics (the forward problem), but here we consider their application for learning the dynamics given the state (the inverse problem). This repurposing of LMMs is largely motivated by growing interest in data-driven modeling of dynamics, but the behavior and analysis of LMMs for discovery turn out to be significantly different from the well-known, existing theory for the forward problem. Assuming a high…
Scuola Superiore Meridionale（南部高等学院）；Università degli Studi di Napoli Federico II（那不勒斯腓特烈二世大学）
We propose a numerical method based on physics-informed Random Projection Neural Networks for the solution of Initial Value Problems (IVPs) of Ordinary Differential Equations (ODEs) with a focus on stiff problems. We address an Extreme Learning Machine with a single hidden layer with radial basis functions having as widths uniformly distributed random variables, while the values of the weights between the input and the hidden layer are set equal to one. The numerical solution of the IVPs is obtained by constructing a system of nonlinear algebraic equations, which is solved with respect to the …
A New Splitting Method for Time-dependent Convection-dominated Diffusion Problems
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Beijing FEGEN Software Co., Ltd.（北京飞egen软件有限公司）；Department of Mathematics, The Chinese University of Hong Kong（香港中文大学数学系）
Learning Hamiltonian Flow Maps from Numerical-Scheme Residuals for Long-Time Multiscale Simulation
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校奥登计算工程与科学研究所）
The Rotman Institute of Philosophy and The Ontario Research Center for Computer Algebra and The School of Mathematical and Statistical Sciences The University of Western Ontario（韦仕敦大学罗特曼哲学研究所、安大略计算机代数研究中心及数学与统计科学学院）；School of Information Technology and Mathematical Sciences University of South Australia（南澳大利亚大学信息技术与数理科学学院）
In numerical time-integration with implicit-explicit (IMEX) methods, a within-step adaptable decomposition called residual balanced decomposition is introduced. With this decomposition, the requirement of a small enough residual in the iterative solver can be removed, consequently, this allows to exchange stability for efficiency. This decomposition transfers any residual occurring in the implicit equation of the implicit-step into the explicit part of the decomposition. By balancing the residual, the accuracy of the local truncation error of the time-stepping method becomes independent from t…
In this paper, we perform the convergence analysis of unsupervised Legendre--Galerkin neural networks (ULGNet), a deep-learning-based numerical method for solving partial differential equations (PDEs). Unlike existing deep learning-based numerical methods for PDEs, the ULGNet expresses the solution as a spectral expansion with respect to the Legendre basis and predicts the coefficients with deep neural networks by solving a variational residual minimization problem. Since the corresponding loss function is equivalent to the residual induced by the linear algebraic system depending on the choic…
Automatic differentiation (AD) is an ensemble of techniques that allow to evaluate accurate numerical derivatives of a mathematical function expressed in a computer programming language. In this paper we use AD for stating and solving solid mechanics problems. Given a finite element discretization of the domain, we evaluate the free energy of the solid as the integral of its strain energy density, and we make use of AD for directly obtaining the residual force vector and the tangent stiffness matrix of the problem, as the gradient and the Hessian of the free energy respectively. The result is …
