We introduce economical versions of standard implicit ODE solvers that are specifically tailored for the efficient and accurate simulation of neural networks. These reformulations allow to achieve a significant increase in the efficiency of network simulations, by reducing the size of the algebraic systems effectively solved at each time step. While we focus here specifically on Explicit first step, Diagonally Implicit Runge Kutta methods (ESDIRK), similar simplifications can also be applied to any implicit ODE solver. In order to demonstrate the capabilities of the proposed methods, we consider networks based on three different single-cell models with slow-fast dynamics, including the classical FitzHugh-Nagumo model, a Intracellular Calcium Concentration model and the Hindmarsh-Rose model. Numerical experiments on the simulation of networks of increasing size based on these models demonstrate the superior efficiency of the proposed economical methods.
We present and analyze a new iterative solver for implicit discretizations of a simplified Boltzmann-Poisson system. The algorithm builds on recent work that incorporated a sweeping algorithm for the Vlasov-Poisson equations as part of nested inner-outer iterative solvers for the Boltzmann-Poisson equations. The new method eliminates the need for nesting and requires only one transport sweep per iteration. It arises as a new fixed-point formulation of the discretized system which we prove to be contractive for a given electric potential. We also derive an accelerator to improve the convergence…
In this work we construct a multiderivative implicit-explicit (IMEX) scheme for a class of stiff ordinary differential equations. Our solver is high-order accurate and has an asymptotic preserving (AP) property. The proposed method is based upon a two-derivative backward Taylor series base solver, which we show has an AP property. Higher order accuracies are found by iterating the result over a high-order multiderivative interpolant of the right hand side function, which we again prove has an AP property. Theoretical results showcasing the asymptotic consistency as well as the high-order accur…
University of Utah（犹他大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Illinois Institute of Technology（伊利诺伊理工学院）；University of California, Irvine（加州大学欧文分校）；Scientific Computing and Imaging (SCI) Institute（科学计算与成像研究所）
Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers. These solvers are computationally expensive, requiring the use of tiny step sizes for numerical stability and accuracy guarantees. This paper considers learning neural ODEs using implicit ODE solvers of different orders leveraging proximal operators. The proximal implicit solver consists of inner-outer iterations: the inner iterations approximate each implicit update step using a fast optimization algorithm, and the outer iterations solve the ODE system over time. The pr…
A novel theoretical convergence rate estimate for a Balancing Domain Decomposition by Constraints algorithm is proven for the solution of the cardiac Bidomain model, describing the propagation of the electric impulse in the cardiac tissue. The non-linear system arises from a fully implicit time discretization and a monolithic solution approach. The preconditioned non-symmetric operator is constructed from the linearized system arising within the Newton-Krylov approach for the solution of the non-linear problem; we theoretically analyze and prove a convergence rate bound for the Generalised Min…
An L-Stable Implicit Two-Stage Fourth-Order Temporal Discretization Scheme for Lax-Wendroff-Type Solvers Applied to Stiff Problems
School of Mathematics and Information Science, Henan Polytechnic University（河南理工大学数学与信息科学学院）；School of Mechatronical Engineering, Beijing Institute of Technology（北京理工大学机械与车辆学院）
Fast and accurate solution of time-dependent partial differential equations (PDEs) is of key interest in many research fields including physics, engineering, and biology. Generally, implicit schemes are preferred over the explicit ones for better stability and correctness. The existing implicit schemes are usually iterative and employ a general-purpose solver which may be sub-optimal for a specific class of PDEs. In this paper, we propose a neural solver to learn an optimal iterative scheme for a class of PDEs, in a data-driven fashion. We attain this objective by modifying an iteration of an …
Technical University Munich（慕尼黑工业大学）；École normale supérieure, Paris（巴黎高等师范学院）；deepc GmbH, Munich（慕尼黑deepc有限公司）；Helmholtz Zentrum München（慕尼黑亥姆霍兹研究中心）；University of Zurich（苏黎世大学）
Fast and accurate solutions of time-dependent partial differential equations (PDEs) are of pivotal interest to many research fields, including physics, engineering, and biology. Generally, implicit/semi-implicit schemes are preferred over explicit ones to improve stability and correctness. However, existing semi-implicit methods are usually iterative and employ a general-purpose solver, which may be sub-optimal for a specific class of PDEs. In this paper, we propose a neural solver to learn an optimal iterative scheme in a data-driven fashion for any class of PDEs. Specifically, we modify a si…
Two novel parallel Newton-Krylov Balancing Domain Decomposition by Constraints (BDDC) and Dual-Primal Finite Element Tearing and Interconnecting (FETI-DP) solvers are here constructed, analyzed and tested numerically for implicit time discretizations of the three-dimensional Bidomain system of equations. This model represents the most advanced mathematical description of the cardiac bioelectrical activity and it consists of a degenerate system of two non-linear reaction-diffusion partial differential equations (PDEs), coupled with a stiff system of ordinary differential equations (ODEs). A fin…
We present a novel fully implicit hybrid finite volume/finite element method for incompressible flows. Following previous works on semi-implicit hybrid FV/FE schemes, the incompressible Navier-Stokes equations are split into a pressure and a transport-diffusion subsystem. The first of them can be seen as a Poisson type problem and is thus solved efficiently using classical continuous Lagrange finite elements. On the other hand, finite volume methods are employed to solve the convective subsystem, in combination with Crouzeix-Raviart finite elements for the discretization of the viscous stress …
Randomized linear system solvers have become popular as they have the potential to reduce floating point complexity while still achieving desirable convergence rates. One particularly promising class of methods, random sketching solvers, has achieved the best known computational complexity bounds in theory, but is blunted by two practical considerations: there is no clear way of choosing the size of the sketching matrix apriori; and there is a nontrivial storage cost of the sketched system. In this work, we make progress towards addressing these issues by implicitly generating the sketched sys…
We use the alternating direction method to simulate implicit dynamics. ur spatial discretization uses isogeometric analysis. Namely, we simulate a (hyperbolic) wave propagation problem in which we use tensor-product B-splines in space and an implicit time marching method to fully discretize the problem. We approximate our discrete operator as a Kronecker product of one-dimensional mass and stiffness matrices. As a result of this algebraic transformation, we can factorize the resulting system of equations in linear (i.e., O(N)) time at each step of the implicit method. We demonstrate the perfor…
Coupled nonlinear system of reaction-diffusion equations describing multi-component (species) interactions with heterogeneous coefficients is considered. Finite volume method based approximation for the space is used to construct semi-discrete form for the computation of numerical solutions. Two techniques for time approximations, namely, a fully implicit (FI) and a semi-implicit (SI) schemes are examined. The fully implicit scheme is constructed using Newton's method and leads to the coupled system of equations on each nonlinear and time iterations which is computationally rather expensive. I…
High-order, linearly stable, partitioned solvers for general multiphysics problems based on implicit-explicit Runge-Kutta schemes
Stanford University（斯坦福大学）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；University of California, Berkeley（加州大学伯克利分校）；University of Notre Dame（圣母大学）
We introduce a fast solver for the phase field crystal (PFC) and functionalized Cahn-Hilliard (FCH) equations with periodic boundary conditions on a rectangular domain that features the preconditioned Nesterov accelerated gradient descent (PAGD) method. We discretize these problems with a Fourier collocation method in space, and employ various second-order schemes in time. We observe a significant speedup with this solver when compared to the preconditioned gradient descent (PGD) method. With the PAGD solver, fully implicit, second-order-in-time schemes are not only feasible to solve the PFC a…
GPU-Accelerated Nested Schur-Complement Solvers for Atmospheric Flow Simulations Using Horizontally-Explicit, Vertically-Implicit Schemes and Discontinuous Galerkin Methods
Johannes Gutenberg University Mainz（约翰内斯·古腾堡美因茨大学）；Universidad Politécnica de Madrid（马德里理工大学）；Leibniz Institute for Tropospheric Research (TROPOS)（莱布尼茨对流层研究所）；University of Cologne（科隆大学）
Differentiate the Solver, Not the Equation: Reverse-Sweep Adjoints for Block Implicit Simulation
Stanford University（斯坦福大学）；University of California, Los Angeles（加州大学洛杉矶分校）；LightSpeed Studios, Tencent America（腾讯美国光速工作室）；The University of Utah（犹他大学）
Radial basis functions are typically used when discretization sche-mes require inhomogeneous node distributions. While spawning from a desire to interpolate functions on a random set of nodes, they have found successful applications in solving many types of differential equations. However, the weights of the interpolated solution, used in the linear superposition of basis functions to interpolate the solution, and the actual value of the solution are completely different. In fact, these weights mix the value of the solution with the geometrical location of the nodes used to discretize the equa…
In this paper, we propose a novel computational implicit method, which we call Isogeometric Residual Minimization (iGRM) with direction splitting. The method mixes the benefits resulting from isogeometric analysis, implicit dynamics, residual minimization, and alternating direction solver. We utilize tensor product B-spline basis functions in space, implicit second order time integration schemes, residual minimization in every time step, and we exploit Kronecker product structure of the matrix to employ linear computational cost alternating direction solver. We implement an implicit time integ…
We focus on non-stationary Maxwell equations defined on a regular patch of elements as considered in the isogeometric analysis (IGA). We apply the time-integration scheme following the ideas developed by the finite difference community [M. Hochbruck, T. Jahnke, R. Schnaubelt, Convergence of an ADI splitting for Maxwell's equations, Numerishe Mathematik, 2015] to derive a weak formulation resulting in discretization with Kronecker product matrices. We take the tensor product structure of the computational patch of elements from the IGA framework as an advantage, allowing for linear computationa…
A comparative study of explicit and implicit Large Eddy Simulations using a high-order discontinuous Galerkin solver: application to a Formula 1 front wing
In numerical time-integration with implicit-explicit (IMEX) methods, a within-step adaptable decomposition called residual balanced decomposition is introduced. With this decomposition, the requirement of a small enough residual in the iterative solver can be removed, consequently, this allows to exchange stability for efficiency. This decomposition transfers any residual occurring in the implicit equation of the implicit-step into the explicit part of the decomposition. By balancing the residual, the accuracy of the local truncation error of the time-stepping method becomes independent from t…
Robust Parameter and State Estimation in Multiscale Neuronal Systems Using Physics-Informed Neural Networks
Department of Mathematics, University of Iowa（数学系，爱荷华大学）；Department of Mathematics, Volen National Center for Complex Systems, Brandeis University（数学系，布兰迪大学复杂系统国家中心）
Numerically solving ordinary differential equations (ODEs) is a naturally serial process and as a result the vast majority of ODE solver software are serial. In this manuscript we developed a set of parallelized ODE solvers using extrapolation methods which exploit "parallelism within the method" so that arbitrary user ODEs can be parallelized. We describe the specific choices made in the implementation of the explicit and implicit extrapolation methods which allow for generating low overhead static schedules to then exploit with optimized multi-threaded implementations. We demonstrate that wh…
A fast neural hybrid Newton solver adapted to implicit methods for nonlinear dynamics
organization= Department of Mathematics , addressline= The Hong Kong University of Science；organization= Mathematical Institute , addressline= University of Oxford , country= United Kingdom；organization= Laboratoire Jacques-Louis Lions (UMR 7598) , addressline= Sorbonne Université , country= France；Autonomous Driving Research Lab , addressline= HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute , city= Futian , state= Shenzhen , country= China
The Cauchy-Kowalewskaya (CK) procedure is a key building block in the design of solvers for the Generalised Rieman Problem (GRP) based on Taylor series expansions in time. The CK procedure allows us to express time derivatives in terms of purely space derivatives. This is a very cumbersome procedure, which often requires the use of software manipulators. In this paper, a simplification of the CK procedure is proposed in the context of implicit Taylor series expansion for GRP, for hyperbolic balance laws in the framework of [Journal of Computational Physics 303 (2015) 146-172]. A recursive form…
This article aims at developing a high order pressure-based solver for the solution of the 3D compressible Navier-Stokes system at all Mach numbers. We propose a cell-centered discretization of the governing equations that splits the fluxes into a fast and a slow scale part, that are treated implicitly and explicitly, respectively. A novel semi-implicit discretization is proposed for the kinetic energy as well as the enthalpy fluxes in the energy equation, hence avoiding any need of iterative solvers. The implicit discretization yields an elliptic equation on the pressure that can be solved fo…
We propose a computational framework for vascular fluid-structure interaction (FSI), focusing on biomechanical modeling, geometric modeling, and solver technology. The biomechanical model is constructed based on the unified continuum formulation. We highlight that the chosen time integration scheme differs from existing implicit FSI integration methods in that it is indeed second-order accurate, does not suffer from the overshoot phenomenon, and optimally dissipates high-frequency modes in both subproblems. We propose a pipeline for generating subject-specific meshes for FSI analysis for anato…
The sequential fully implicit (SFI) scheme was introduced (Jenny et al. 2006) for solving coupled flow and transport problems. Each time step for SFI consists of an outer loop, in which there are inner Newton loops to implicitly and sequentially solve the pressure and transport sub-problems. In standard SFI, the sub-problems are usually solved with tight tolerances at every outer iteration. This can result in wasted computations that contribute little progress towards the coupled solution. The issue is known as `over-solving'. Our objective is to minimize the cost of inner solvers while mainta…
