GMRES using pseudoinverse for range symmetric singular systems
Kota Sugihara, Ken Hayami, Liao Zeyu
备注：
Sentence at end of Section 1 when rhs contains discretization, measurement errors. Section 2 on motivation. Theorem 4.1: necessary, sufficient conditions for inconsistent, consistent cases. After Theorem 4.1, difference between theory and experiments explained. Modified Definition 2. Eliminated results for plat1919, saylr3. Modified Conclusions. References 1,2,3 on applications
发表信息：
Journal of Computational and Applied Mathematics, 422 (2023) 114865
Consider solving large sparse range symmetric singular linear systems $ A {\bf x}= {\bf b} $ which arise, for instance, in the discretization of convection diffusion equations with periodic boundary conditions, and partial differential equations for electromagnetic fields using the edge-based finite element method.
In theory, the Generalized Minimal Residual (GMRES) method converges to the least squares solution for inconsistent systems if the coefficient matrix $A$ is range symmetric, i.e. $ {\rm R}(A)= {\rm R}(A^{ \rm T } )$, where $ {\rm R}(A)$ is the range space of $A$.
We derived the necessary and sufficient conditions for GMRES to determine a least squares solution of inconsistent and consistent range symmetric systems assuming exact arithmetic except for the computation of the elements of the Hessenberg matrix.
In practice, GMRES may not converge due to numerical instability. In order to improve the convergence, we propose using the pseudoinverse for the solution of the severely ill-conditioned Hessenberg systems in GMRES. Numerical experiments on inconsistent systems indicate that the method is effective and robust. Finally, we further improve the convergence of the method by reorthogonalizing the Modified Gram-Schmidt procedure.
We consider finite difference approximations of the second derivative, exemplified in Poisson's equation, the heat equation and the wave equation. The finite difference operators satisfy a summation-by-parts property, which mimics the integration-by-parts. Since the operators approximate the second derivative, they are singular by construction. To impose boundary conditions, these operators are modified using Simultaneous Approximation Terms. This makes the modified matrices non-singular, for most choices of boundary conditions. Recently, inverses of such matrices were derived. However, when c…
Consider using the right-preconditioned GMRES (AB-GMRES) for obtaining the minimum-norm solution of inconsistent underdetermined systems of linear equations. Morikuni (Ph.D. thesis, 2013) showed that for some inconsistent and ill-conditioned problems, the iterates may diverge. This is mainly because the Hessenberg matrix in the GMRES method becomes very ill-conditioned so that the backward substitution of the resulting triangular system becomes numerically unstable. We propose a stabilized GMRES based on solving the normal equations corresponding to the above triangular system using the standa…
We introduce a new class of preconditioners to enable flexible GMRES to find a least-squares solution, and potentially the pseudoinverse solution, of large-scale sparse, asymmetric, singular, and potentially inconsistent systems. We develop the preconditioners based on a new observation that generalized inverses (i.e., $\boldsymbol{A}^{g}\in\{\boldsymbol{G}\mid\boldsymbol{A}\boldsymbol{G}\boldsymbol{A}=\boldsymbol{A}\}$) enable the preconditioned Krylov subspaces to converge in a single step. We then compute an approximate generalized inverse (AGI) efficiently using a hybrid incomplete factori…
In [Hayami K, Sugihara M. Numer Linear Algebra Appl. 2011; 18:449--469], the authors analyzed the convergence behaviour of the Generalized Minimal Residual (GMRES) method for the least squares problem $ \min_{ {\bf x} \in {\bf R}^n} {\| {\bf b} - A {\bf x} \|_2}^2$, where $ A \in {\bf R}^{n \times n}$ may be singular and $ {\bf b} \in {\bf R}^n$, by decomposing the algorithm into the range $ {\cal R}(A) $ and its orthogonal complement $ {\cal R}(A)^\perp $ components. However, we found that the proof of the fact that GMRES gives a least squares solution if $ {\cal R}(A) = {\cal R}(A^{\scriptsi…
This paper presents novel methodologies for the numerical simulation of scattering of elastic waves by both closed and open surfaces in three-dimensional space. The proposed approach utilizes new integral formulations as well as an extension to the elastic context of the efficient high-order singular-integration methods~\cite{BG18} introduced recently for the acoustic case. In order to obtain formulations leading to iterative solvers (GMRES) which converge in small numbers of iterations we investigate, theoretically and computationally, the character of the spectra of various operators associa…
CG, SYMMLQ, and MINRES are Krylov subspace methods for solving symmetric systems of linear equations. When these methods are applied to an incompatible system (that is, a singular symmetric least-squares problem), CG could break down and SYMMLQ's solution could explode, while MINRES would give a least-squares solution but not necessarily the minimum-length (pseudoinverse) solution. This understanding motivates us to design a MINRES-like algorithm to compute minimum-length solutions to singular symmetric systems. MINRES uses QR factors of the tridiagonal matrix from the Lanczos process (where R…
It is known that the solution of a conservative steady-state two-sided fractional diffusion problem can exhibit singularities near the boundaries. As consequence of this, and due to the conservative nature of the problem, we adopt a finite volume elements discretization approach over a generic non-uniform mesh. We focus on grids mapped by a smooth function which consist in a combination of a graded mesh near the singularity and a uniform mesh where the solution is smooth. Such a choice gives rise to Toeplitz-like discretization matrices and thus allows a low computational cost of the matrix-ve…
In our previous work [SIAM J. Sci. Comput. 43(3) (2021) B784-B810], an accurate hyper-singular boundary integral equation method for dynamic poroelasticity in two dimensions has been developed. This work is devoted to studying the more complex and difficult three-dimensional problems with Neumann boundary condition and both the direct and indirect methods are adopted to construct combined boundary integral equations. The strongly-singular and hyper-singular integral operators are reformulated into compositions of weakly-singular integral operators and tangential-derivative operators, which all…
We introduce a software package called HIFIR for preconditioning sparse, unsymmetric, ill-conditioned, and potentially singular systems. HIFIR computes a hybrid incomplete factorization, which combines multilevel incomplete LU factorization with a truncated, rank-revealing QR factorization on the final Schur complement. This novel hybridization is based on the new theory of approximate generalized inverse and $ε$-accuracy. It enables near-optimal preconditioners for consistent systems and enables flexible GMRES to solve inconsistent systems when coupled with iterative refinement. In this paper…
We present an accelerated and hardware parallelized integral-equation solver for the problem of acoustic scattering by a two-dimensional surface in three-dimensional space. The approach is based, in part, on the novel Interpolated Factored Green Function acceleration method (IFGF) that, without recourse to the Fast Fourier Transform (FFT), evaluates the action of Green function-based integral operators for an $N$-point surface discretization at a complexity of $\Ord(N\log N)$ operations instead of the $\Ord(N^2)$ cost associated with nonaccelerated methods. The IFGF algorithm exploits the slow…
Parameter-free inexact block Schur complement preconditioning for linear poroelasticity under a hybrid Bernardi-Raugel and weak Galerkin finite element discretization
