The problem of optimal precision switching for the conjugate gradient (CG) method applied to sparse linear systems is considered. A sparse matrix is defined as an $n\!\times\!n$ matrix with $m\!=\!O(n)$ nonzero entries. The algorithm first computes an approximate solution in single precision with tolerance $\varepsilon_1$, then switches to double precision to refine the solution to the required stopping tolerance $\varepsilon_2$. Based on estimates of system matrix parameters -- computed in time which does not exceed $1\%$ of the time needed to solve the system in double precision -- we determine the optimal value of $\varepsilon_1$ that minimizes total computation time. This value is obtained by classifying the matrix using the $k$-nearest neighbors method on a small precomputed sample. Classification relies on a feature vector comprising: the matrix size $n$, the number of nonzeros $m$, the pseudo-diameter of the matrix sparsity graph, and the average rate of residual norm decay during the early CG iterations in single precision. We show that, in addition to the matrix condition number, the diameter of the sparsity graph influences the growth of rounding errors during iterative computations. The proposed algorithm reduces the computational complexity of the CG -- expressed in equivalent double-precision iterations -- by more than $17\%$ on average across the considered matrix types in a sequential setting. The resulting speedup is at most $1.5\%$ worse than that achieved with the optimal (oracle) choice of $\varepsilon_1$.
While the impact of matrix structure on Krylov subspace method convergence is well understood, the use of the sparsity graph diameter as a predictive feature for rounding error growth in mixed-precision CG appears to be novel. To the best of our knowledge, no prior work employs graph diameter to guide precision switching in iterative linear solvers.
Restarted Hessenberg method for solving shifted nonsymmetric linear systems
University of Electronic Science and Technology of China（电子科技大学）；University of Groningen（格罗宁根大学）；Sun Yat-sen University（中山大学）；Nottingham Trent University（诺丁汉特伦特大学）；Dalian University of Technology（大连理工大学）
We present a fast iterative solver for scattering problems in 2D, where a penetrable object with compact support is considered. By representing the scattered field as a volume potential in terms of the Green's function, we arrive at the Lippmann-Schwinger equation in integral form, which is then discretized using an appropriate quadrature technique. The discretized linear system is then solved using an iterative solver accelerated by Directional Algebraic Fast Multipole Method (DAFMM). The DAFMM presented here relies on the directional admissibility condition of the 2D Helmholtz kernel. And th…
We study computational complexity aspects for Finite Element formulations considering hypercubic space--time full and time--marching discretization schemes for $h$--refined grids towards singularities. We perform a relatively comprehensive study of comparing the computational time via time complexities of direct and iterative solvers. We focus on the space-time formulation with refined computational grids and on the corresponding time slabs, namely, computational grids obtained by taking the "cross-sections" of the refined space-time mesh. We compare the computational complexity of the space-t…
This paper presents the results of a preliminary experimental investigation of the performance of a stationary iterative method based on a block staircase splitting for solving singular systems of linear equations arising in Markov chain modelling. From the experiments presented, we can deduce that the method is well suited for solving block banded or more generally localized systems in a parallel computing environment. The parallel implementation has been benchmarked using several Markovian models.
Deterministic and randomized, row-action and column-action linear solvers have become increasingly popular owing to their simplicity, low computational and memory complexities, and ease of composition with other techniques. Moreover, in order to achieve high-performance, such solvers must often be adapted to the given problem structure and to the hardware platform on which the problem will be solved. Unfortunately, determining whether such adapted solvers will converge to a solution has required equally unique analyses. As a result, adapted, reliable solvers are slow to be developed and deploy…
We consider the goal-oriented error estimates for a linearized iterative solver for nonlinear partial differential equations. For the adjoint problem and iterative solver we consider, instead of the differentiation of the primal problem, a suitable linearization which guarantees the adjoint consistency of the numerical scheme. We derive error estimates and develop an efficient adaptive algorithm which balances the errors arising from the discretization and use of iterative solvers. Several numerical examples demonstrate the efficiency of this algorithm.
This paper introduces a discretization-accurate stopping criterion of symmetric iterative methods for solving systems of algebraic equations resulting from the finite element approximation. The stopping criterion consists of the evaluations of the discretization and the algebraic error estimators, that are based on the respective duality error estimator and the difference of two consecutive iterates. Iterations are terminated when the algebraic estimator is of the same magnitude as the discretization estimator. Numerical results for multigrid $V(1,1)$-cycle and symmetric Gauss-Seidel iterative…
In this paper, we develop a (preconditioned) GMRES solver based on integer arithmetic, and introduce an iterative refinement framework for the solver. We describe the data format for the coefficient matrix and vectors for the solver that is based on integer or fixed-point numbers. To avoid overflow in calculations, we introduce initial scaling and logical shifts (adjustments) of operands in arithmetic operations. We present the approach for operand shifts, considering the characteristics of the GMRES algorithm. Numerical tests demonstrate that the integer arithmetic-based solver with iterative…
The discovery of fast numerical solvers prompted a clear and rapid shift towards iterative techniques in many applications, especially in computational mechanics, due to the increased necessity for solving very large linear systems. Most numerical solvers are highly dependent on the problem geometry and discretization, facing issues when any of these properties change. The newly developed Hybrid Iterative Numerical Transferable Solver (HINTS) combines a standard solver with a neural operator to achieve better performance, focusing on a single geometry at a time. In this work, we explore the "T…
We present a scalable and efficient iterative solver for high-order hybridized discontinuous Galerkin (HDG) discretizations of hyperbolic partial differential equations. It is an interplay between domain decomposition methods and HDG discretizations. In particular, the method is a fixed-point approach that requires only independent element-by-element local solves in each iteration. As such, it is well-suited for current and future computing systems with massive concurrencies. We rigorously show that the proposed method is exponentially convergent in the number of iterations for transport and l…
In this thesis, the numerical solution of three different classes of problems have been studied. Specifically, new techniques have been proposed and their theoretical analysis has been performed, accompanied by a wide set of numerical experiments, for investigating further and comparing the effectiveness and performance of the presented approach. The first two belong to the research area of numerical linear algebra and concern the spectral analysis and preconditioning for Krylov subspace methods of the coefficient matrix of large structured linear systems. The third concerns a problem from the…
We present and analyze a new iterative solver for implicit discretizations of a simplified Boltzmann-Poisson system. The algorithm builds on recent work that incorporated a sweeping algorithm for the Vlasov-Poisson equations as part of nested inner-outer iterative solvers for the Boltzmann-Poisson equations. The new method eliminates the need for nesting and requires only one transport sweep per iteration. It arises as a new fixed-point formulation of the discretized system which we prove to be contractive for a given electric potential. We also derive an accelerator to improve the convergence…
We present a scalability study of Golub-Kahan bidiagonalization for the parallel iterative solution of symmetric indefinite linear systems with a 2x2 block structure. The algorithms have been implemented within the parallel numerical library PETSc. Since a nested inner-outer iteration strategy may be necessary, we investigate different choices for the inner solvers, including parallel sparse direct and multigrid accelerated iterative methods. We show the strong and weak scalability of the Golub-Kahan bidiagonalization based iterative method when applied to a two-dimensional Poiseuille flow and…
This manuscript presents an efficient solver for the linear system that arises from the Hierarchical Poincaré-Steklov (HPS) discretization of three dimensional variable coefficient Helmholtz problems. Previous work on the HPS method has tied it with a direct solver. This work is the first efficient iterative solver for the linear system that results from the HPS discretization. The solution technique utilizes GMRES coupled with a locally homogenized block-Jacobi preconditioner. The local nature of the discretization and preconditioner naturally yield the matrix-free application of the linear s…
Randomized linear system solvers have become popular as they have the potential to reduce floating point complexity while still achieving desirable convergence rates. One particularly promising class of methods, random sketching solvers, has achieved the best known computational complexity bounds in theory, but is blunted by two practical considerations: there is no clear way of choosing the size of the sketching matrix apriori; and there is a nontrivial storage cost of the sketched system. In this work, we make progress towards addressing these issues by implicitly generating the sketched sys…
We consider the application of the WaveHoltz iteration to time-harmonic elastic wave equations with energy conserving boundary conditions. The original WaveHoltz iteration for acoustic Helmholtz problems is a fixed-point iteration that filters the solution of the wave equation with time-harmonic forcing and boundary data. As in the original WaveHoltz method, we reformulate the fixed point iteration as a positive definite linear system of equations that is iteratively solved by a Krylov method. We present two time-stepping schemes, one explicit and one (novel) implicit, which completely remove …
Solving linear systems is a ubiquitous task in science and engineering. Because directly inverting a large-scale linear system can be computationally expensive, iterative algorithms are often used to numerically find the inverse. To accommodate the dynamic range and precision requirements, these iterative algorithms are often carried out on floating-point processing units. Low-precision, fixed-point processors require only a fraction of the energy per operation consumed by their floating-point counterparts, yet their current usages exclude iterative solvers due to the computational errors aris…
We consider a second-order elliptic boundary value problem with strongly monotone and Lipschitz-continuous nonlinearity. We design and study its adaptive numerical approximation interconnecting a finite element discretization, the Banach-Picard linearization, and a contractive linear algebraic solver. We in particular identify stopping criteria for the algebraic solver that on the one hand do not request an overly tight tolerance but on the other hand are sufficient for the inexact (perturbed) Banach-Picard linearization to remain contractive. Similarly, we identify suitable stopping criteria …
$O(N)$ Iterative and $O(NlogN)$ Fast Direct Volume Integral Equation Solvers with a Minimal-Rank ${\cal H}^2$-Representation for Large-Scale $3$-D Electrodynamic Analysis
School of Electrical and Computer Engineering, Purdue University（普渡大学电气与计算机工程学院）
Isogeometric analysis for 2D and 3D curl-div problems: Spectral symbols and fast iterative solvers
Max-Planck Institut für Plasmaphysik（马克斯·普朗克等离子体物理研究所）；University of Rome “Tor Vergata”（罗马托维加塔大学）；Technische Universität München（慕尼黑工业大学）；University of Insubria（因苏布里亚大学）；Uppsala University（乌普萨拉大学）
We present a family of discretizations for the Variable Eddington Factor (VEF) equations that have high-order accuracy on curved meshes and efficient preconditioned iterative solvers. The VEF discretizations are combined with a high-order Discontinuous Galerkin transport discretization to form an effective high-order, linear transport method. The VEF discretizations are derived by extending the unified analysis of Discontinuous Galerkin methods for elliptic problems to the VEF equations. This framework is used to define analogs of the interior penalty, second method of Bassi and Rebay, minimal…
Frequency-domain full-waveform inversion (FWI) is suitable for long-offset stationary-recording acquisition, since reliable subsurface models can be reconstructed with a few frequencies and attenuation is easily implemented without computational overhead. In the frequency domain, wave modelling is a Helmholtz-type boundary-value problem which requires to solve a large and sparse system of linear equations per frequency with multiple right-hand sides (sources). This system can be solved with direct or iterative methods. While the former are suitable for FWI application on 3D dense OBC acquisiti…
Large sparse linear algebraic systems can be found in a variety of scientific and engineering fields, and many scientists strive to solve them in an efficient and robust manner. In this paper, we propose an interpretable neural solver, the Fourier Neural Solver (FNS), to address them. FNS is based on deep learning and Fast Fourier transform. Because the error between the iterative solution and the ground truth involves a wide range of frequency modes, FNS combines a stationary iterative method and frequency space correction to eliminate different components of the error. Local Fourier analysis…
There are some types of ill-conditioned algebraic equations that have difficulty in obtaining accurate roots and coefficients that must be expressed with a multiple precision floating-point number. When all their roots are simple, the problem solved via eigensolver (eigenvalue method) is well-conditioned if the corresponding companion matrix has its small condition number. However, directly solving them with Newton or simultaneous iteration methods (direct iterative method for short) should be considered as ill-conditioned because of increasing density of its root distribution. Although a grea…
