Deflation techniques for Krylov subspace methods have seen a lot of attention in recent years. They provide means to improve the convergence speed of these methods by enriching the Krylov subspace with a deflation subspace. The most common approach for the construction of deflation subspaces is to use (approximate) eigenvectors, but also more general subspaces are applicable.
In this paper we discuss two results concerning the accuracy requirements within the deflated CG method. First we show that the effective condition number which bounds the convergence rate of the deflated conjugate gradient method depends asymptotically linearly on the size of the perturbations in the deflation subspace. Second, we discuss the accuracy required in calculating the deflating projection. This is crucial concerning the overall convergence of the method, and also allows to save some computational work.
To show these results, we use the fact that as a projection approach deflation has many similarities to multigrid methods. In particular, recent results relate the spectra of the deflated matrix to the spectra of the error propagator of twogrid methods. In the spirit of these results we show that the effective condition number can be bounded by the constant of a weak approximation property.
In this paper, we extend to the block case, the a posteriori bound showing superlinear convergence of Conjugate Gradients developed in [J. Comput. Applied Math., 48 (1993), pp. 327-341]; that is, we obtain similar bounds, but now for block Conjugate Gradients. We also present a series of computational experiments illustrating the validity of the bound developed here, as well as the bound from [SIAM Review, 47 (2005), pp. 247-272] using angles between subspaces. Using these bounds, we make some observations on the onset of superlinearity, and how this onset depends on the eigenvalue distributio…
We analyze the convergence of the Conjugate Gradient (CG) method in exact arithmetic, when the coefficient matrix $A$ is symmetric positive semidefinite and the system is consistent. To do so, we diagonalize $A$ and decompose the algorithm into the range and the null space components of $A$. Further, we apply the analysis to the CGLS and CGNE (CG Normal Error) methods for rank-deficient least squares problems.
Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method
Toulouse-INP (ENSEEIHT)/IRIT（图卢兹INP（ENSEEIHT）/IRIT）；Artificial and Natural Intelligence Toulouse Institute (ANITI)（人工智能与自然智能图卢兹研究所（ANITI））；Division of Applied Mathematics, Brown University（应用数学系，布朗大学）
In the framework of inverse linear problems on infinite-dimensional Hilbert space, we prove the convergence of the conjugate gradient iterates to an exact solution to the inverse problem in the most general case where the self-adjoint, non-negative operator is unbounded and with minimal, technically unavoidable assumptions on the initial guess of the iterative algorithm. The convergence is proved to always hold in the Hilbert space norm (error convergence), as well as at other levels of regularity (energy norm, residual, etc.) depending on the regularity of the iterates. We also discuss, both …
An important class of nonlinear weighted least-squares problems arises from the assimilation of observations in atmospheric and ocean models. In variational data assimilation, inverse error covariance matrices define the weighting matrices of the least-squares problem. For observation errors, a diagonal matrix (i.e., uncorrelated errors) is often assumed for simplicity even when observation errors are suspected to be correlated. While accounting for observationerror correlations should improve the quality of the solution, it also affects the convergence rate of the minimization algorithms used…
In this note we exploit polynomial preconditioners for the Conjugate Gradient method to solve large symmetric positive definite linear systems in a parallel environment. We put in connection a specialized Newton method to solve the matrix equation X^{-1} = A and the Chebyshev polynomials for preconditioning. We propose a simple modification of one parameter which avoids clustering of extremal eigenvalues in order to speed-up convergence. We provide results on very large matrices (up to 8 billion unknowns) in a parallel environment showing the efficiency of the proposed class of preconditioners…
On the convergence of orthogonalization-free conjugate gradient method for extreme eigenvalues of Hermitian matrices: a Riemannian optimization interpretation
Connections of the conjugate gradient (CG) method with other methods in computational mathematics are surveyed, including the connections with the conjugate direction method, the subspace optimization method and the quasi-Newton method BFGS in numrical optimization, and the Lanczos method in numerical linear algebra. Two sequences of polynomials related to residual vectors and conjugate vectors are reviewed, where the residual polynomials are similar to orthogonal polynomials in the approximation theory and the roots of the polynomials reveal certain information of the coefficient matrix. The …
Alternating least squares (ALS) is often considered the workhorse algorithm for computing the rank-R canonical tensor approximation, but for certain problems its convergence can be very slow. The nonlinear conjugate gradient (NCG) method was recently proposed as an alternative to ALS, but the results indicated that NCG is usually not faster than ALS. To improve the convergence speed of NCG, we consider a nonlinearly preconditioned nonlinear conjugate gradient (PNCG) algorithm for computing the rank-R canonical tensor decomposition. Our approach uses ALS as a nonlinear preconditioner in the NCG…
Structure preserving quaternion conjugate gradient-type methods for solving non-Hermitian quaternion linear systems
School of Mathematics and Statistics, Key Laboratory of Analytical Mathematics and Applications (Ministry of Education), Fujian Normal University（福建师范大学数学与统计学院，分析数学及应用教育部重点实验室）；School of Mathematics and Statistics, Hainan University（海南大学数学与统计学院）；School of Mathematical Sciences, South China Normal University（华南师范大学数学科学学院）
We consider the conjugate gradient algorithm applied to a general class of spiked sample covariance matrices. The main result of the paper is that the norms of the error and residual vectors at any finite step concentrate on deterministic values determined by orthogonal polynomials with respect to a deformed Marchenko--Pastur law. The first-order limits and fluctuations are shown to be universal. Additionally, for the case where the bulk eigenvalues lie in a single interval we show a stronger universality result in that the asymptotic rate of convergence of the conjugate gradient algorithm onl…
Solving systems of linear equations is a problem occuring frequently in water engineering applications. Usually the size of the problem is too large to be solved via direct factorization. One can resort to iterative approaches, in particular the conjugate gradients method if the matrix is symmetric positive definite. Preconditioners further enhance the rate of convergence but hitherto only handcrafted ones requiring expert knowledge have been used. We propose an innovative approach employing Machine Learning, in particular a Convolutional Neural Network, to unassistedly design preconditioning …
Efficient spectral computation of the stationary states of rotating Bose-Einstein condensates by the preconditioned nonlinear conjugate gradient method
Université de Lorraine（洛林大学）；Inria Nancy-Grand Est（法国国家信息与自动化研究院南锡-大东部中心）；Inria Paris（法国国家信息与自动化研究院巴黎中心）；Université Paris-Est, CERMICS (ENPC)（巴黎东大学，CERMICS（国立路桥学校））；Laboratoire de Mathématiques Raphaël Salem, Université de Rouen（鲁昂大学拉斐尔·萨莱姆数学实验室）
A fundamental task in numerical computation is the solution of large linear systems. The conjugate gradient method is an iterative method which offers rapid convergence to the solution, particularly when an effective preconditioner is employed. However, for more challenging systems a substantial error can be present even after many iterations have been performed. The estimates obtained in this case are of little value unless further information can be provided about the numerical error. In this paper we propose a novel statistical model for this numerical error set in a Bayesian framework. Our…
