The convergence of GMRES for solving linear systems can be influenced heavily by the structure of the right hand side. Within the solution of eigenvalue problems via inverse iteration or subspace iteration, the right hand side is generally related to an approximate invariant subspace of the linear system. We give detailed and new bounds on (block) GMRES that take the special behavior of the right hand side into account and explain the initial sharp decrease of the GMRES residual. The bounds give rise to adapted preconditioners applied to the eigenvalue problems, e.g. tuned and polynomial preconditioners. The numerical results show that the new (block) GMRES bounds are much sharper than conventional bounds and that preconditioned subspace iteration with either a tuned or polynomial preconditioner should be used in practice.
It is well-established that any non-increasing convergence curve is possible for GMRES and a family of pairs $(A,b)$ can be constructed for which GMRES exhibits a given convergence curve with $A$ having arbitrary spectrum. No analog of this result has been established for block GMRES, wherein multiple right-hand sides are considered. By reframing the problem as a single linear system over a ring of square matrices, we develop convergence results for block Arnoldi and block GMRES. In particular, we show what convergence behavior is admissible for block GMRES and how the matrices and right-hand …
GMRES is a popular Krylov subspace method for solving linear systems of equations involving a general non-Hermitian coefficient matrix. The conventional bounds on GMRES convergence involve polynomial approximation problems in the complex plane. Three popular approaches pose this approximation problem on the spectrum, the field of values, or pseudospectra of the coefficient matrix. We analyze and compare these bounds, illustrating with six examples the success and failure of each. When the matrix departs from normality due only to a low-dimensional invariant subspace, we discuss how these bound…
Suppose that a linear bounded operator $B$ on a Hilbert space exhibits at least linear GMRES convergence, i.e., there exists $M_B<1$ such that the GMRES residuals fulfill $\|r_k\|\leq M_B\|r_{k-1}\|$ for every initial residual $r_0$ and step $k\in\mathbb{N}$. We prove that GMRES with a compactly perturbed operator $A=B+C$ admits the bound $\|r_k\|/\|r_0\|\leq\prod_{j=1}^k\bigl(M_B+(1+M_B)\,\|A^{-1}\|\,σ_j(C)\bigr)$, i.e., the singular values $σ_j(C)$ control the departure from the bound for the unperturbed problem. This result can be seen as an extension of [I. Moret, A note on the superlinear…
Department of Mathematics and Computational Modeling and Data Analytics Division, Academy of Integrated Science, Virginia Tech（弗吉尼亚理工大学综合科学学院数学与计算建模及数据分析部）；Department of Mathematics, Baylor University（贝勒大学数学系）；Department of Mathematics, Computer Science, and Cooperative Engineering, University of St. Thomas（圣托马斯大学数学、计算机科学与合作工程系）
In many applications, linear systems arise where the coefficient matrix takes the special form ${\bf I} + {\bf K} + {\bf E}$, where ${\bf I}$ is the identity matrix of dimension $n$, ${\rm rank}({\bf K}) = p \ll n$, and $\|{\bf E}\| \leq ε< 1$. GMRES convergence rates for linear systems with coefficient matrices of the forms ${\bf I} + {\bf K}$ and ${\bf I} + {\bf E}$ are guaranteed by well-known theory, but only relatively weak convergence bounds specific to matrices of the form ${\bf I} + {\bf K} + {\bf E}$ currently exist. In this paper, we explore the convergence properties of linear syste…
This paper develops a new class of algorithms for general linear systems and eigenvalue problems. These algorithms apply fast randomized sketching to accelerate subspace projection methods, such as GMRES and Rayleigh--Ritz. This approach offers great flexibility in designing the basis for the approximation subspace, which can improve scalability in many computational environments. The resulting algorithms outperform the classic methods with minimal loss of accuracy. For model problems, numerical experiments show large advantages over MATLAB's optimized routines, including a $100 \times$ speedu…
With the commercial availability of mixed precision hardware, mixed precision GMRES-based iterative refinement schemes have emerged as popular approaches for solving sparse linear systems. Existing analyses of these approaches, however, are based on using full LU factorizations to construct preconditioners for use within GMRES in each refinement step. In practical applications, inexact preconditioning techniques, such as incomplete LU or sparse approximate inverses, are often used for performance reasons. In this work, we investigate the use of sparse approximate inverse preconditioners based …
We consider nonlinear convergence acceleration methods for fixed-point iteration $x_{k+1}=q(x_k)$, including Anderson acceleration (AA), nonlinear GMRES (NGMRES), and Nesterov-type acceleration (corresponding to AA with window size one). We focus on fixed-point methods that converge asymptotically linearly with convergence factor $ρ<1$ and that solve an underlying fully smooth and non-convex optimization problem. It is often observed that AA and NGMRES substantially improve the asymptotic convergence behavior of the fixed-point iteration, but this improvement has not been quantified theoretica…
One of the main tools for solving linear systems arising from the discretization of the Helmholtz equation is the shifted Laplace preconditioner, which results from the discretization of a perturbed Helmholtz problem $-Δu - (k^2 + i \varepsilon )u = f$ where $0 \neq \varepsilon \in \mathbb{R}$ is an absorption parameter. In this work we revisit the idea of combining the shifted Laplace preconditioner with two-level deflation and apply it to Helmholtz problems discretized with linear finite elements. We use the convergence theory of GMRES based on the field of values to prove that GMRES applied…
We consider GMRES applied to discretisations of the high-frequency Helmholtz equation with strong trapping; recall that in this situation the problem is exponentially ill-conditioned through an increasing sequence of frequencies. Under certain assumptions about the distribution of the eigenvalues, we prove upper bounds on how the number of GMRES iterations grows with the frequency. Our main focus is on boundary-integral-equation formulations of the exterior Dirichlet and Neumann obstacle problems in 2- and 3-d; for these problems, we investigate numerically the sharpness (in terms of dependenc…
We present a polynomial preconditioner for solving large systems of linear equations. The polynomial is derived from the minimum residual polynomial (the GMRES polynomial) and is more straightforward to compute and implement than many previous polynomial preconditioners. Our current implementation of this polynomial using its roots is naturally more stable than previous methods of computing the same polynomial. We implement further stability control using added roots, and this allows for high degree polynomials. We discuss the effectiveness and challenges of root-adding and give an additional …
Steepest descent preconditioning is considered for the recently proposed nonlinear generalized minimal residual (N-GMRES) optimization algorithm for unconstrained nonlinear optimization. Two steepest descent preconditioning variants are proposed. The first employs a line search, while the second employs a predefined small step. A simple global convergence proof is provided for the N-GMRES optimization algorithm with the first steepest descent preconditioner (with line search), under mild standard conditions on the objective function and the line search processes. Steepest descent preconditioni…
We consider linear parameter-dependent systems $A(μ) x(μ) = b$ for many different $μ$, where $A$ is large and sparse, and depends nonlinearly on $μ$. Solving such systems individually for each $μ$ would require great computational effort. In this work we propose to compute a partial parameterization $\tilde{x} \approx x(μ)$ where $\tilde{x}(μ)$ is cheap to compute for many different $μ$. Our methods are based on the observation that a companion linearization can be formed where the dependence on $μ$ is only linear. In particular, we develop methods which combine the well-established Krylov sub…
Multilevel Preconditioner with Stable Coarse Grid Corrections for the Helmholtz Equation
Xiamen University（厦门大学）；Nanjing University（南京大学）；Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院计算数学与科学工程计算研究所）
On $τ$-preconditioners for a quasi-compact difference scheme to Riesz fractional diffusion equations with variable coefficients
Hanshan Normal University（韩山师范学院）；Southwestern University of Finance and Economics（西南财经大学）；University of Insubria（因苏布里亚大学）；University of Uppsala（乌普萨拉大学）
We propose a preconditioner to accelerate the convergence of the GMRES iterative method for solving the system of linear equations obtained from discretize-then-optimize approach applied to optimal control problems constrained by a partial differential equation. Eigenvalue distribution of the preconditioned matrix as well as its eigenvectors are discussed. Numerical results of the proposed preconditioner are compared with several existing preconditioners to show its efficiency.
