In the framework of inverse linear problems on infinite-dimensional Hilbert space, we prove the convergence of the conjugate gradient iterates to an exact solution to the inverse problem in the most general case where the self-adjoint, non-negative operator is unbounded and with minimal, technically unavoidable assumptions on the initial guess of the iterative algorithm. The convergence is proved to always hold in the Hilbert space norm (error convergence), as well as at other levels of regularity (energy norm, residual, etc.) depending on the regularity of the iterates. We also discuss, both analytically and through a selection of numerical tests, the main features and differences of our convergence result as compared to the case, already available in the literature, where the operator is bounded.
We prove convergence rates of monotone schemes for conservation laws for Hölder continuous initial data with unbounded total variation, provided that the Hölder exponent of the initial data is greater than $1/2$. For strictly $\mathrm{Lip}^+$ stable monotone schemes, we prove convergence for any positive Hölder exponent. Numerical experiments are presented which verify the theory.
In this paper, we extend to the block case, the a posteriori bound showing superlinear convergence of Conjugate Gradients developed in [J. Comput. Applied Math., 48 (1993), pp. 327-341]; that is, we obtain similar bounds, but now for block Conjugate Gradients. We also present a series of computational experiments illustrating the validity of the bound developed here, as well as the bound from [SIAM Review, 47 (2005), pp. 247-272] using angles between subspaces. Using these bounds, we make some observations on the onset of superlinearity, and how this onset depends on the eigenvalue distributio…
We analyze the convergence of the Conjugate Gradient (CG) method in exact arithmetic, when the coefficient matrix $A$ is symmetric positive semidefinite and the system is consistent. To do so, we diagonalize $A$ and decompose the algorithm into the range and the null space components of $A$. Further, we apply the analysis to the CGLS and CGNE (CG Normal Error) methods for rank-deficient least squares problems.
The abstract issue of 'Krylov solvability' is extensively discussed for the inverse problem $Af = g$ where $A$ is a (possibly unbounded) linear operator on an infinite-dimensional Hilbert space, and $g$ is a datum in the range of $A$. The question consists of whether the solution $f$ can be approximated in the Hilbert norm by finite linear combinations of $g, Ag, A^2 g, \dots$ , and whether solutions of this sort exist and are unique. After revisiting the known picture when $A$ is bounded, we study the general case of a densely defined and closed $A$. Intrinsic operator-theoretic mechanisms ar…
This paper is concerned with the time-dependent acoustic-elastic interaction problem associated with a bounded elastic body immersed in a homogeneous air or fluid above an unbounded rough surface. The well-posedness and stability of the problem are first established by using the Laplace transform and the energy method. A perfectly matched layer (PML) is then introduced to truncate the interaction problem above a finite layer containing the elastic body, leading to a PML problem in a finite strip domain. We further establish the existence, uniqueness and stability estimate of solutions to the P…
On the convergence of orthogonalization-free conjugate gradient method for extreme eigenvalues of Hermitian matrices: a Riemannian optimization interpretation
An important class of nonlinear weighted least-squares problems arises from the assimilation of observations in atmospheric and ocean models. In variational data assimilation, inverse error covariance matrices define the weighting matrices of the least-squares problem. For observation errors, a diagonal matrix (i.e., uncorrelated errors) is often assumed for simplicity even when observation errors are suspected to be correlated. While accounting for observationerror correlations should improve the quality of the solution, it also affects the convergence rate of the minimization algorithms used…
We consider the numerical algorithm for the two-dimensional time-harmonic elastic wave scattering by unbounded rough surfaces with Dirichlet boundary condition. A Nyström method is proposed for the scattering problem based on the integral equation method. Convergence of the Nyström method is established with convergence rate depending on the smoothness of the rough surfaces. In doing so, a crucial role is played by analyzing the singularities of the kernels of the relevant boundary integral operators. Numerical experiments are presented to demonstrate the effectiveness of the method.
In this article, we develop and analyze a full discretization, based on the spatial spectral Galerkin method and the temporal drift implicit Euler scheme, for the stochastic Cahn--Hilliard equation driven by multiplicative space-time white noise. By introducing an appropriate decomposition of the numerical approximation, we first use the factorization method to deduce the a priori estimate and regularity estimate of the proposed full discretization. With the help of the variation approach, we then obtain the sharp spatial and temporal convergence rate in negative Sobolev space in mean square s…
Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method
Toulouse-INP (ENSEEIHT)/IRIT（图卢兹INP（ENSEEIHT）/IRIT）；Artificial and Natural Intelligence Toulouse Institute (ANITI)（人工智能与自然智能图卢兹研究所（ANITI））；Division of Applied Mathematics, Brown University（应用数学系，布朗大学）
Alternating least squares (ALS) is often considered the workhorse algorithm for computing the rank-R canonical tensor approximation, but for certain problems its convergence can be very slow. The nonlinear conjugate gradient (NCG) method was recently proposed as an alternative to ALS, but the results indicated that NCG is usually not faster than ALS. To improve the convergence speed of NCG, we consider a nonlinearly preconditioned nonlinear conjugate gradient (PNCG) algorithm for computing the rank-R canonical tensor decomposition. Our approach uses ALS as a nonlinear preconditioner in the NCG…
In this note we exploit polynomial preconditioners for the Conjugate Gradient method to solve large symmetric positive definite linear systems in a parallel environment. We put in connection a specialized Newton method to solve the matrix equation X^{-1} = A and the Chebyshev polynomials for preconditioning. We propose a simple modification of one parameter which avoids clustering of extremal eigenvalues in order to speed-up convergence. We provide results on very large matrices (up to 8 billion unknowns) in a parallel environment showing the efficiency of the proposed class of preconditioners…
Connections of the conjugate gradient (CG) method with other methods in computational mathematics are surveyed, including the connections with the conjugate direction method, the subspace optimization method and the quasi-Newton method BFGS in numrical optimization, and the Lanczos method in numerical linear algebra. Two sequences of polynomials related to residual vectors and conjugate vectors are reviewed, where the residual polynomials are similar to orthogonal polynomials in the approximation theory and the roots of the polynomials reveal certain information of the coefficient matrix. The …
Solving systems of linear equations is a problem occuring frequently in water engineering applications. Usually the size of the problem is too large to be solved via direct factorization. One can resort to iterative approaches, in particular the conjugate gradients method if the matrix is symmetric positive definite. Preconditioners further enhance the rate of convergence but hitherto only handcrafted ones requiring expert knowledge have been used. We propose an innovative approach employing Machine Learning, in particular a Convolutional Neural Network, to unassistedly design preconditioning …
