The SPAI algorithm, a sparse approximate inverse preconditioning technique for large sparse linear systems, proposed by Grote and Huckle [SIAM J. Sci. Comput., 18 (1997), pp.~838--853.], is based on the F-norm minimization and computes a sparse approximate inverse $M$ of a large sparse matrix $A$ adaptively. However, SPAI may be costly to seek the most profitable indices at each loop and $M$ may be ineffective for preconditioning. In this paper, we propose a residual based sparse approximate inverse preconditioning procedure (RSAI), which, unlike SPAI, is based on only the {\em dominant} rather than all information on the current residual and augments sparsity patterns adaptively during the loops. RSAI is less costly to seek indices and is more effective to capture a good approximate sparsity pattern of $A^{-1}$ than SPAI. To control the sparsity of $M$ and reduce computational cost, we develop a practical RSAI($tol$) algorithm that drops small nonzero entries adaptively during the process. Numerical experiments are reported to demonstrate that RSAI($tol$) is at least competitive with SPAI and can be considerably more efficient and effective than SPAI. They also indicate that RSAI($tol$) is comparable to the PSAI($tol$) algorithm proposed by one of the authors in 2009.
These are lectures notes on support preconditioning, originally written in 2007. The notes cover large sparse linear systems of equations, krylov subspace solvers for symmetric diagonally dominant linear systems, sparse Cholesky, support theory, Laplacians, and augmented maximum-spanning tree preconditioning.
With the commercial availability of mixed precision hardware, mixed precision GMRES-based iterative refinement schemes have emerged as popular approaches for solving sparse linear systems. Existing analyses of these approaches, however, are based on using full LU factorizations to construct preconditioners for use within GMRES in each refinement step. In practical applications, inexact preconditioning techniques, such as incomplete LU or sparse approximate inverses, are often used for performance reasons. In this work, we investigate the use of sparse approximate inverse preconditioners based …
This work proposes an original preconditioner that couples the Constrained Pressure Residual (CPR) method with block preconditioning for the efficient solution of the linearized systems of equations arising from fully implicit multiphase flow models. This preconditioner, denoted as Block CPR (BCPR), is specifically designed for Lagrange multipliers-based flow models, such as those generated by Mixed Hybrid Finite Element (MHFE) approximations. An original MHFE-based formulation of the two-phase flow model is taken as a reference for the development of the BCPR preconditioner, in which the set …
We present two minimum residual methods for solving sequences of shifted linear systems, the right-preconditioned shifted GMRES and shifted recycled GMRES algorithms which use a seed projection strategy often employed to solve multiple related problems. These methods are compatible with general preconditioning of all systems, and when restricted to right preconditioning, require no extra applications of the operator or preconditioner. These seed projection methods perform a minimum residual iteration for the base system while improving the approximations for the shifted systems at little addit…
A generalized sampling and preconditioning scheme for sparse approximation of polynomial chaos expansions
Computer Science Research Institute, Sandia National Laboratories（桑迪亚国家实验室计算机科学研究所）；Scientific Computing and Imaging (SCI) Institute and Mathematics Department, University of Utah（犹他大学科学计算与成像研究所及数学系）；Institute of Computational Mathematics and the Chinese Academy of Sciences（中国科学院计算数学研究所）
We establish a new iterative method for solving a class of large and sparse linear systems of equations with three-by-three block coefficient matrices having saddle point structure. Convergence properties of the proposed method are studied in details and its induced preconditioner is examined for accelerating the convergence speed of generalized minimal residual (GMRES) method. More precisely, we analyze the eigenvalue distribution of the preconditioned matrix. Numerical experiments are reported to demonstrate the effectiveness of the proposed preconditioner.
We propose domain decomposition preconditioners for the solution of an integral equation formulation of forward and inverse acoustic scattering problems with point scatterers. We study both forward and inverse problems and propose preconditioning techniques to accelerate the iterative solvers. For the forward scattering problem, we extend the domain decomposition based preconditioning techniques presented for partial differential equations in {\em "A restricted additive Schwarz preconditioner for general sparse linear systems", SIAM Journal on Scientific Computing, 21 (1999), pp. 792--797}, to…
Iterative solvers preconditioned with algebraic multigrid have been devised as an optimal technology to speed up the response of large sparse linear systems. In this work, this technique was implemented in the framework of the dual delineation approach. This involves a single groundwater flow solve and a pure advective transport solve with different right-hand sides. The new solver was compared with traditional preconditioned iterative methods and direct sparse solvers on several two- and three-dimensional benchmark problems spanning homogeneous and heterogeneous formations. For the groundwate…
This paper discusses parGeMSLR, a C++/MPI software library for the solution of sparse systems of linear algebraic equations via preconditioned Krylov subspace methods in distributed-memory computing environments. The preconditioner implemented in parGeMSLR is based on algebraic domain decomposition and partitions the symmetrized adjacency graph recursively into several non-overlapping partitions via a p-way vertex separator, where p is an integer multiple of the total number of MPI processes. From a numerical perspective, parGeMSLR builds a Schur complement approximate inverse preconditioner a…
Parameter-free inexact block Schur complement preconditioning for linear poroelasticity under a hybrid Bernardi-Raugel and weak Galerkin finite element discretization
Polynomial preconditioning with the GMRES minimal residual polynomial has the potential to greatly reduce orthogonalization costs, making it useful for communication reduction. We implement polynomial preconditioning in the Belos package from Trilinos and show how it can be effective in both serial and parallel implementations. We further show it is a communication-avoiding technique and is a viable option to CA-GMRES for large-scale parallel computing.
The paper focuses on developing and studying efficient block preconditioners based on classical algebraic multigrid for the large-scale sparse linear systems arising from the fully coupled and implicitly cell-centered finite volume discretization of multi-group radiation diffusion equations, whose coefficient matrices can be rearranged into the $(G+2)\times(G+2)$ block form, where $G$ is the number of energy groups. The preconditioning techniques are based on the monolithic classical algebraic multigrid method, physical-variable based coarsening two-level algorithm and two types of block Schur…
In this paper, we propose a simple sparse approximate inverse for triangular matrices (SAIT). Using the Jacobi iteration method, we obtain an expression of the exact inverse of triangular matrix, which is a finite series. The SAIT is constructed based on this series. We apply the SAIT matrices to iterative methods with ILU preconditioners. The two triangular solvers in the ILU preconditioning procedure are replaced by two matrix-vector multiplications, which can be fine-grained parallelized. We test this method by solving some linear systems and eigenvalue problems with preconditioned iterativ…
Preconditioning of a coupled Cahn--Hilliard Navier--Stokes system
The University of British Columbia（不列颠哥伦比亚大学）；Technical University München（慕尼黑工业大学）；Max Planck Institute for Dynamics of Complex Technical Systems（马克斯·普朗克复杂技术系统动力学研究所）
Steepest descent preconditioning is considered for the recently proposed nonlinear generalized minimal residual (N-GMRES) optimization algorithm for unconstrained nonlinear optimization. Two steepest descent preconditioning variants are proposed. The first employs a line search, while the second employs a predefined small step. A simple global convergence proof is provided for the N-GMRES optimization algorithm with the first steepest descent preconditioner (with line search), under mild standard conditions on the objective function and the line search processes. Steepest descent preconditioni…
We introduce a new class of preconditioners to enable flexible GMRES to find a least-squares solution, and potentially the pseudoinverse solution, of large-scale sparse, asymmetric, singular, and potentially inconsistent systems. We develop the preconditioners based on a new observation that generalized inverses (i.e., $\boldsymbol{A}^{g}\in\{\boldsymbol{G}\mid\boldsymbol{A}\boldsymbol{G}\boldsymbol{A}=\boldsymbol{A}\}$) enable the preconditioned Krylov subspaces to converge in a single step. We then compute an approximate generalized inverse (AGI) efficiently using a hybrid incomplete factori…
This paper presents a parallel preconditioning approach based on incomplete LU (ILU) factorizations in the framework of Domain Decomposition (DD) for general sparse linear systems. We focus on distributed memory parallel architectures, specifically, those that are equipped with graphic processing units (GPUs). In addition to block Jacobi, we present general purpose two-level ILU Schur complement-based approaches, where different strategies are presented to solve the coarse-level reduced system. These strategies are combined with modified ILU methods in the construction of the coarse-level oper…
Nonlinear Preconditioning: How to use a Nonlinear Schwarz Method to Precondition Newton's Method
University of Strathclyde（斯特拉斯克莱德大学）；Université de Genève（日内瓦大学）；Université Nice Sophia Antipolis（尼斯索菲亚安蒂波利斯大学）；Inria（法国国家数字科学研究所）；Hong Kong Baptist University（香港浸会大学）
Sparse residual tree (SRT) is an adaptive exploration method for multivariate scattered data approximation. It leads to sparse and stable approximations in areas where the data is sufficient or redundant, and points out the possible local regions where data refinement is needed. Sparse residual forest (SRF) is a combination of SRT predictors to further improve the approximation accuracy and stability according to the error characteristics of SRTs. The hierarchical parallel SRT algorithm is based on both tree decomposition and adaptive radial basis function (RBF) explorations, whereby for each …
We present a novel stagewise strategy for improving greedy algorithms for sparse recovery. We demonstrate its efficiency both for synthesis and analysis sparse priors, where in both cases we demonstrate its computational efficiency and competitive reconstruction accuracy. In the synthesis case, we also provide theoretical guarantees for the signal recovery that are on par with the existing perfect reconstruction bounds for the relaxation-based solvers and other sophisticated greedy algorithms.
In this paper, we develop a (preconditioned) GMRES solver based on integer arithmetic, and introduce an iterative refinement framework for the solver. We describe the data format for the coefficient matrix and vectors for the solver that is based on integer or fixed-point numbers. To avoid overflow in calculations, we introduce initial scaling and logical shifts (adjustments) of operands in arithmetic operations. We present the approach for operand shifts, considering the characteristics of the GMRES algorithm. Numerical tests demonstrate that the integer arithmetic-based solver with iterative…
