In classical frameworks as the Euclidean space, positive definite kernels as well as their analytic properties are explicitly available and can be incorporated directly in kernel-based learning algorithms. This is different if the underlying domain is a discrete irregular graph. In this case, respective kernels have to be computed in a preliminary step in order to apply them inside a kernel machine. Typically, such a kernel is given as a matrix function of the graph Laplacian. Its direct calculation leads to a high computational burden if the size of the graph is very large. In this work, we investigate five different block Krylov subspace methods to obtain cheaper iterative approximations of these kernels. We will investigate convergence properties of these Krylov subspace methods and study to what extent these methods are able to preserve the symmetry and positive definiteness of the original kernels they are approximating. We will further discuss the computational complexity and the memory requirements of these methods, as well as possible implications for the kernel predictors in machine learning.
Conjugated gradients on the normal equation (CGNE) is a popular method to regularise linear inverse problems. The idea of the method can be summarised as minimising the residuum over a suitable Krylov subspace. It is shown that using the same idea for the shift-and-invert rational Krylov subspace yields an order-optimal regularisation scheme.
In this paper, by introducing a class of relaxed filtered Krylov subspaces, we propose the relaxed filtered Krylov subspace method for computing the eigenvalues with the largest real parts and the corresponding eigenvectors of non-symmetric matrices. As by-products, the generalizations of the filtered Krylov subspace method and the Chebyshev-Davidson method for solving non-symmetric eigenvalue problems are also presented. We give the convergence analysis of the complex Chebyshev polynomial, which plays a significant role in the polynomial acceleration technique. In addition, numerical experime…
Subspace recycling iterative methods and other subspace augmentation schemes are a successful extension to Krylov subspace methods in which a Krylov subspace is augmented with a fixed subspace spanned by vectors deemed to be helpful in accelerating convergence or conveying knowledge of the solution. Recently, a survey was published, in which a framework describing the vast majority of such methods was proposed [Soodhalter et al, GAMM-Mitt. 2020]. In many of these methods, the Krylov subspace is one generated by the system matrix composed with a projector that depends on the augmentation space.…
Randomized orthogonal projection methods (ROPMs) can be used to speed up the computation of Krylov subspace methods in various contexts. Through a theoretical and numerical investigation, we establish that these methods produce quasi-optimal approximations over the Krylov subspace. Our numerical experiments outline the convergence of ROPMs for all matrices in our test set, with occasional spikes, but overall with a convergence rate similar to that of standard OPMs.
The accumulated quadrature weights of Gaussian quadrature formulae constitute bounds on the integral over the intervals between the quadrature nodes. Classical results in this concern date back to works of Chebyshev, Markov and Stieltjes and are referred to as Separation Theorem of Chebyshev-Markov-Stieltjes (CMS Theorem). Similar separation theorems hold true for some classes of rational Gaussian quadrature. The Krylov subspace for a given matrix and initial vector is closely related to orthogonal polynomials associated with the spectral distribution of the initial vector in the eigenbasis of…
A general framework for oblique projections of nonhermitian matrices onto rational Krylov subspaces is developed. To obtain this framework we revisit the classical rational Krylov subspace algorithm and prove that the projected matrix can be written efficiently as a structured pencil, where the structure can take several forms, such as Hessenberg or inverse Hessenberg. One specific instance of the structures appearing in this framework for oblique projections is a tridiagonal pencil. This is a direct generalization of the classical biorthogonal Krylov subspace method where the projection becom…
The use of block Krylov subspace methods for computing the solution to a sequence of shifted linear systems using subspace recycling was first proposed in [Soodhalter, SISC 2016], where a recycled shifted block GMRES algorithm (rsbGMRES) was proposed. Such methods use the equivalence of the shifted system to a Sylvester equation and exploit the shift invariance of the block Krylov subspace generated from the Sylvester operator. This avoids the need for initial residuals to span the same subspace and allows for a viable restarted Krylov subspace method with recycling for solving sequences of sh…
In this work, we propose a reduced basis method for efficient solution of parametric linear systems. The coefficient matrix is assumed to be a linear matrix-valued function that is symmetric and positive definite for admissible values of the parameter $\mathbfσ\in \mathbb{R}^s$. We propose a solution strategy where one first computes a basis for the appropriate compound Krylov subspace and then uses this basis to compute a subspace solution for multiple $\mathbfσ$. Three kinds of compound Krylov subspaces are discussed. Error estimate is given for the subspace solution from each of these space…
In the present paper, we introduce new tensor Krylov subspace methods for solving linear tensor equations. The proposed methods use the well known T-product for tensors and tensor subspaces related to tube fibers. We introduce some new tensor products and the related algebraic properties. These new products will enable us to develop third-order the tensor tubal GMRES and the tensor tubal Golub Kahan methods. We give some properties related to these methods and proopse some numerical experiments.
The second-order cone linear complementarity problem (SOCLCP) is a generalization of the classical linear complementarity problem. It has been known that SOCLCP, with the globally uniquely solvable property, is essentially equivalent to a zero-finding problem in which the associated function bears much similarity to the transfer function in model reduction [{\em SIAM J. Sci. Comput.}, 37 (2015), pp.~A2046--A2075]. In this paper, we propose a new rational Krylov subspace method to solve the zero-finding problem for the symmetric and positive definite SOCLCP. The algorithm consists of two stages…
This paper presents a new algorithm KIOPS for computing linear combinations of $φ$-functions that appear in exponential integrators. This algorithm is suitable for large-scale problems in computational physics where little or no information about the spectrum or norm of the Jacobian matrix is known a priori. We first show that such problems can be solved efficiently by computing a single exponential of a modified matrix. Then our approach is to compute an appropriate basis for the Krylov subspace using the incomplete orthogonalization procedure and project the matrix exponential on this subspa…
It is a well-known fact that the Krylov space $\mathcal{K}_j(H,x)$ generated by a skew-Hamiltonian matrix $H \in \mathbb{R}^{2n \times 2n}$ and some $x \in \mathbb{R}^{2n}$ is isotropic for any $j \in \mathbb{N}$. For any given isotropic subspace $\mathcal{L} \subset \mathbb{R}^{2n}$ of dimension $n$ - which is called a Lagrangian subspace - the question whether $\mathcal{L}$ can be generated as the Krylov space of some skew-Hamiltonian matrix is considered. The affine variety $\mathbb{HK}$ of all skew-Hamiltonian matrices $H \in \mathbb{R}^{2n \times 2n}$ that generate $\mathcal{L}$ as a Kryl…
Many Krylov subspace methods for shifted linear systems take advantage of the invariance of the Krylov subspace under a shift of the matrix. However, exploiting this fact in the non-Hermitian case introduces restrictions; e.g., initial residuals must be collinear and this collinearity must be maintained at restart. Thus we cannot simultaneously solve shifted systems with unrelated right-hand sides using this strategy, and all shifted residuals cannot be simultaneously minimized over a Krylov subspace such that collinearity is maintained. It has been shown that this renders them generally incom…
An algorithm for constructing a $J$-orthogonal basis of the extended Krylov subspace $\mathcal{K}_{r,s}=\operatorname{range}\{u,Hu, H^2u,$ $ \ldots, $ $H^{2r-1}u, H^{-1}u, H^{-2}u, \ldots, H^{-2s}u\},$ where $H \in \mathbb{R}^{2n \times 2n}$ is a large (and sparse) Hamiltonian matrix is derived (for $r = s+1$ or $r=s$). Surprisingly, this allows for short recurrences involving at most five previously generated basis vectors. Projecting $H$ onto the subspace $\mathcal{K}_{r,s}$ yields a small Hamiltonian matrix. The resulting HEKS algorithm may be used in order to approximate $f(H)u$ where $f$ …
This paper deals with the definition and optimization of augmentation spaces for faster convergence of the conjugate gradient method in the resolution of sequences of linear systems. Using advanced convergence results from the literature, we present a procedure based on a selection of relevant approximations of the eigenspaces for extracting, selecting and reusing information from the Krylov subspaces generated by previous solutions in order to accelerate the current iteration. Assessments of the method are proposed in the cases of both linear and nonlinear structural problems.
Block Krylov subspace methods (KSMs) comprise building blocks in many state-of-the-art solvers for large-scale matrix equations as they arise, e.g., from the discretization of partial differential equations. While extended and rational block Krylov subspace methods provide a major reduction in iteration counts over polynomial block KSMs, they also require reliable solvers for the coefficient matrices, and these solvers are often iterative methods themselves. It is not hard to devise scenarios in which the available memory, and consequently the dimension of the Krylov subspace, is limited. In s…
We evaluate the performance of the Krylov subspace method by using highly efficient multiple precision sparse matrix-vector multiplication (SpMV). BNCpack is our multiple precision numerical computation library based on MPFR/GMP, which is one of the most efficient arbitrary precision floating-point arithmetic libraries. However, it does not include functions that can manipulate multiple precision sparse matrices. Therefore, by using benchmark tests, we show that SpMV implemented in these functions can be more efficient. Finally, we also show that product-type Krylov subspace methods such as Bi…
The FEAST eigenvalue algorithm is a subspace iteration algorithm that uses contour integration in the complex plane to obtain the eigenvectors of a matrix for the eigenvalues that are located in any user-defined search interval. By computing small numbers of eigenvalues in specific regions of the complex plane, FEAST is able to naturally parallelize the solution of eigenvalue problems by solving for multiple eigenpairs simultaneously. The traditional FEAST algorithm is implemented by directly solving collections of shifted linear systems of equations; in this paper, we describe a variation of …
The Fréchet derivative $L_f(A,E)$ of the matrix function $f(A)$ plays an important role in many different applications, including condition number estimation and network analysis. We present several different Krylov subspace methods for computing low-rank approximations of $L_f(A,E)$ when the direction term $E$ is of rank one (which can easily be extended to general low-rank). We analyze the convergence of the resulting method for the important special case that $A$ is Hermitian and $f$ is either the exponential, the logarithm or a Stieltjes function. In a number of numerical tests, both inclu…
The present paper is concerned with developing tensor iterative Krylov subspace methods to solve large multi-linear tensor equations. We use the well-known T-product for two tensors to define tensor global Arnoldi and tensor global Gloub-Kahan bidiagonalization algorithms. Furthermore, we illustrate how tensor-based global approaches can be exploited to solve ill-posed problems arising from recovering blurry multichannel (color) images and videos, using the so-called Tikhonov regularization technique, to provide computable approximate regularized solutions. We also review a generalized cross-v…
A generalized matrix Krylov subspace method for TV regularization
Faculté des Sciences et Techniques-Gueliz, Laboratoire de Mathématiques Appliquées et Informatique（盖利兹科学技术学院，应用数学与计算机实验室）；Université de Lille Nord de France, L.M.P.A, ULCO（法国北部里尔大学，L.M.P.A，ULCO）
We propose a time-exact Krylov-subspace-based method for solving linear ODE (ordinary differential equation) systems of the form $y'=-Ay + g(t)$, where $y(t)$ is the unknown function. The method consists of two stages. The first stage is an accurate polynomial approximation of the source term $g(t)$, constructed with the help of the truncated SVD (singular value decomposition). The second stage is a special residual-based block Krylov subspace method. The accuracy of the method is only restricted by the accuracy of the polynomial approximation and by the error of the block Krylov process. Sinc…
Krylov subspace recycling is a powerful tool for solving long series of large, sparse linear systems that change slowly. In PDE constrained shape optimization, these appear naturally, as hundreds or more optimization steps are needed with only small changes in the geometry. In this setting, however, applying Krylov subspace recycling can be difficult. As the geometry evolves, so does the finite element mesh, especially if re-meshing is needed. As a result, the number of algebraic degrees of freedom in the system may change from one optimization step to the next, and with it the size of the fin…
During the past decade, Model Order Reduction (MOR) has become key enabler for the efficient simulation of large circuit models. MOR techniques based on moment-matching are well established due to their simplicity and computational performance in the reduction process. However, moment-matching methods based on the ordinary Krylov subspace are usually inadequate to accurately approximate the original circuit behavior. In this paper, we present a moment-matching method which is based on the extended Krylov subspace and exploits the superposition property in order to deal with many terminals. The…
