A standard approach to model reduction of large-scale higher-order linear dynamical systems is to rewrite the system as an equivalent first-order system and then employ Krylov-subspace techniques for model reduction of first-order systems. This paper presents some results about the structure of the block-Krylov subspaces induced by the matrices of such equivalent first-order formulations of higher-order systems. Two general classes of matrices, which exhibit the key structures of the matrices of first-order formulations of higher-order systems, are introduced. It is proved that for both classes, the block-Krylov subspaces induced by the matrices in these classes can be viewed as multiple copies of certain subspaces of the state space of the original higher-order system.
Conjugated gradients on the normal equation (CGNE) is a popular method to regularise linear inverse problems. The idea of the method can be summarised as minimising the residuum over a suitable Krylov subspace. It is shown that using the same idea for the shift-and-invert rational Krylov subspace yields an order-optimal regularisation scheme.
In this paper, by introducing a class of relaxed filtered Krylov subspaces, we propose the relaxed filtered Krylov subspace method for computing the eigenvalues with the largest real parts and the corresponding eigenvectors of non-symmetric matrices. As by-products, the generalizations of the filtered Krylov subspace method and the Chebyshev-Davidson method for solving non-symmetric eigenvalue problems are also presented. We give the convergence analysis of the complex Chebyshev polynomial, which plays a significant role in the polynomial acceleration technique. In addition, numerical experime…
Subspace recycling iterative methods and other subspace augmentation schemes are a successful extension to Krylov subspace methods in which a Krylov subspace is augmented with a fixed subspace spanned by vectors deemed to be helpful in accelerating convergence or conveying knowledge of the solution. Recently, a survey was published, in which a framework describing the vast majority of such methods was proposed [Soodhalter et al, GAMM-Mitt. 2020]. In many of these methods, the Krylov subspace is one generated by the system matrix composed with a projector that depends on the augmentation space.…
Randomized orthogonal projection methods (ROPMs) can be used to speed up the computation of Krylov subspace methods in various contexts. Through a theoretical and numerical investigation, we establish that these methods produce quasi-optimal approximations over the Krylov subspace. Our numerical experiments outline the convergence of ROPMs for all matrices in our test set, with occasional spikes, but overall with a convergence rate similar to that of standard OPMs.
The accumulated quadrature weights of Gaussian quadrature formulae constitute bounds on the integral over the intervals between the quadrature nodes. Classical results in this concern date back to works of Chebyshev, Markov and Stieltjes and are referred to as Separation Theorem of Chebyshev-Markov-Stieltjes (CMS Theorem). Similar separation theorems hold true for some classes of rational Gaussian quadrature. The Krylov subspace for a given matrix and initial vector is closely related to orthogonal polynomials associated with the spectral distribution of the initial vector in the eigenbasis of…
A general framework for oblique projections of nonhermitian matrices onto rational Krylov subspaces is developed. To obtain this framework we revisit the classical rational Krylov subspace algorithm and prove that the projected matrix can be written efficiently as a structured pencil, where the structure can take several forms, such as Hessenberg or inverse Hessenberg. One specific instance of the structures appearing in this framework for oblique projections is a tridiagonal pencil. This is a direct generalization of the classical biorthogonal Krylov subspace method where the projection becom…
The use of block Krylov subspace methods for computing the solution to a sequence of shifted linear systems using subspace recycling was first proposed in [Soodhalter, SISC 2016], where a recycled shifted block GMRES algorithm (rsbGMRES) was proposed. Such methods use the equivalence of the shifted system to a Sylvester equation and exploit the shift invariance of the block Krylov subspace generated from the Sylvester operator. This avoids the need for initial residuals to span the same subspace and allows for a viable restarted Krylov subspace method with recycling for solving sequences of sh…
In this work, we propose a reduced basis method for efficient solution of parametric linear systems. The coefficient matrix is assumed to be a linear matrix-valued function that is symmetric and positive definite for admissible values of the parameter $\mathbfσ\in \mathbb{R}^s$. We propose a solution strategy where one first computes a basis for the appropriate compound Krylov subspace and then uses this basis to compute a subspace solution for multiple $\mathbfσ$. Three kinds of compound Krylov subspaces are discussed. Error estimate is given for the subspace solution from each of these space…
In the present paper, we introduce new tensor Krylov subspace methods for solving linear tensor equations. The proposed methods use the well known T-product for tensors and tensor subspaces related to tube fibers. We introduce some new tensor products and the related algebraic properties. These new products will enable us to develop third-order the tensor tubal GMRES and the tensor tubal Golub Kahan methods. We give some properties related to these methods and proopse some numerical experiments.
The second-order cone linear complementarity problem (SOCLCP) is a generalization of the classical linear complementarity problem. It has been known that SOCLCP, with the globally uniquely solvable property, is essentially equivalent to a zero-finding problem in which the associated function bears much similarity to the transfer function in model reduction [{\em SIAM J. Sci. Comput.}, 37 (2015), pp.~A2046--A2075]. In this paper, we propose a new rational Krylov subspace method to solve the zero-finding problem for the symmetric and positive definite SOCLCP. The algorithm consists of two stages…
This paper presents a new algorithm KIOPS for computing linear combinations of $φ$-functions that appear in exponential integrators. This algorithm is suitable for large-scale problems in computational physics where little or no information about the spectrum or norm of the Jacobian matrix is known a priori. We first show that such problems can be solved efficiently by computing a single exponential of a modified matrix. Then our approach is to compute an appropriate basis for the Krylov subspace using the incomplete orthogonalization procedure and project the matrix exponential on this subspa…
It is a well-known fact that the Krylov space $\mathcal{K}_j(H,x)$ generated by a skew-Hamiltonian matrix $H \in \mathbb{R}^{2n \times 2n}$ and some $x \in \mathbb{R}^{2n}$ is isotropic for any $j \in \mathbb{N}$. For any given isotropic subspace $\mathcal{L} \subset \mathbb{R}^{2n}$ of dimension $n$ - which is called a Lagrangian subspace - the question whether $\mathcal{L}$ can be generated as the Krylov space of some skew-Hamiltonian matrix is considered. The affine variety $\mathbb{HK}$ of all skew-Hamiltonian matrices $H \in \mathbb{R}^{2n \times 2n}$ that generate $\mathcal{L}$ as a Kryl…
Many Krylov subspace methods for shifted linear systems take advantage of the invariance of the Krylov subspace under a shift of the matrix. However, exploiting this fact in the non-Hermitian case introduces restrictions; e.g., initial residuals must be collinear and this collinearity must be maintained at restart. Thus we cannot simultaneously solve shifted systems with unrelated right-hand sides using this strategy, and all shifted residuals cannot be simultaneously minimized over a Krylov subspace such that collinearity is maintained. It has been shown that this renders them generally incom…
An algorithm for constructing a $J$-orthogonal basis of the extended Krylov subspace $\mathcal{K}_{r,s}=\operatorname{range}\{u,Hu, H^2u,$ $ \ldots, $ $H^{2r-1}u, H^{-1}u, H^{-2}u, \ldots, H^{-2s}u\},$ where $H \in \mathbb{R}^{2n \times 2n}$ is a large (and sparse) Hamiltonian matrix is derived (for $r = s+1$ or $r=s$). Surprisingly, this allows for short recurrences involving at most five previously generated basis vectors. Projecting $H$ onto the subspace $\mathcal{K}_{r,s}$ yields a small Hamiltonian matrix. The resulting HEKS algorithm may be used in order to approximate $f(H)u$ where $f$ …
This paper deals with the definition and optimization of augmentation spaces for faster convergence of the conjugate gradient method in the resolution of sequences of linear systems. Using advanced convergence results from the literature, we present a procedure based on a selection of relevant approximations of the eigenspaces for extracting, selecting and reusing information from the Krylov subspaces generated by previous solutions in order to accelerate the current iteration. Assessments of the method are proposed in the cases of both linear and nonlinear structural problems.
Block Krylov subspace methods (KSMs) comprise building blocks in many state-of-the-art solvers for large-scale matrix equations as they arise, e.g., from the discretization of partial differential equations. While extended and rational block Krylov subspace methods provide a major reduction in iteration counts over polynomial block KSMs, they also require reliable solvers for the coefficient matrices, and these solvers are often iterative methods themselves. It is not hard to devise scenarios in which the available memory, and consequently the dimension of the Krylov subspace, is limited. In s…
We evaluate the performance of the Krylov subspace method by using highly efficient multiple precision sparse matrix-vector multiplication (SpMV). BNCpack is our multiple precision numerical computation library based on MPFR/GMP, which is one of the most efficient arbitrary precision floating-point arithmetic libraries. However, it does not include functions that can manipulate multiple precision sparse matrices. Therefore, by using benchmark tests, we show that SpMV implemented in these functions can be more efficient. Finally, we also show that product-type Krylov subspace methods such as Bi…
We propose a time-exact Krylov-subspace-based method for solving linear ODE (ordinary differential equation) systems of the form $y'=-Ay + g(t)$, where $y(t)$ is the unknown function. The method consists of two stages. The first stage is an accurate polynomial approximation of the source term $g(t)$, constructed with the help of the truncated SVD (singular value decomposition). The second stage is a special residual-based block Krylov subspace method. The accuracy of the method is only restricted by the accuracy of the polynomial approximation and by the error of the block Krylov process. Sinc…
The FEAST eigenvalue algorithm is a subspace iteration algorithm that uses contour integration in the complex plane to obtain the eigenvectors of a matrix for the eigenvalues that are located in any user-defined search interval. By computing small numbers of eigenvalues in specific regions of the complex plane, FEAST is able to naturally parallelize the solution of eigenvalue problems by solving for multiple eigenpairs simultaneously. The traditional FEAST algorithm is implemented by directly solving collections of shifted linear systems of equations; in this paper, we describe a variation of …
During the past decade, Model Order Reduction (MOR) has become key enabler for the efficient simulation of large circuit models. MOR techniques based on moment-matching are well established due to their simplicity and computational performance in the reduction process. However, moment-matching methods based on the ordinary Krylov subspace are usually inadequate to accurately approximate the original circuit behavior. In this paper, we present a moment-matching method which is based on the extended Krylov subspace and exploits the superposition property in order to deal with many terminals. The…
An efficient Krylov subspace algorithm for computing actions of the $φ$ matrix function for large matrices is proposed. This matrix function is widely used in exponential time integration, Markov chains and network analysis and many other applications. Our algorithm is based on a reliable residual based stopping criterion and a new efficient restarting procedure. For matrices with numerical range in the stable complex half plane, we analyze residual convergence and prove that the restarted method is guaranteed to converge for any Krylov subspace dimension. Numerical tests demonstrate efficienc…
In this paper a new restarting method for Krylov subspace matrix exponential evaluations is proposed. Since our restarting technique essentially employs the residual, some convergence results for the residual are given. We also discuss how the restart length can be adjusted after each restart cycle, which leads to an adaptive restarting procedure. Numerical tests are presented to compare our restarting with three other restarting methods. Some of the algorithms described in this paper are a part of the Octave/Matlab package expmARPACK available at http://team.kiam.ru/botchev/expm/.
The Fréchet derivative $L_f(A,E)$ of the matrix function $f(A)$ plays an important role in many different applications, including condition number estimation and network analysis. We present several different Krylov subspace methods for computing low-rank approximations of $L_f(A,E)$ when the direction term $E$ is of rank one (which can easily be extended to general low-rank). We analyze the convergence of the resulting method for the important special case that $A$ is Hermitian and $f$ is either the exponential, the logarithm or a Stieltjes function. In a number of numerical tests, both inclu…
New algorithms are proposed for the Tucker approximation of a 3-tensor, that access it using only the tensor-by-vector-by-vector multiplication subroutine. In the matrix case, Krylov methods are methods of choice to approximate the dominant column and row subspaces of a sparse or structured matrix given through the matrix-by-vector multiplication subroutine. Using the Wedderburn rank reduction formula, we propose an algorithm of matrix approximation that computes Krylov subspaces and allows generalization to the tensor case. Several variants of proposed tensor algorithms differ by pivoting str…
In classical frameworks as the Euclidean space, positive definite kernels as well as their analytic properties are explicitly available and can be incorporated directly in kernel-based learning algorithms. This is different if the underlying domain is a discrete irregular graph. In this case, respective kernels have to be computed in a preliminary step in order to apply them inside a kernel machine. Typically, such a kernel is given as a matrix function of the graph Laplacian. Its direct calculation leads to a high computational burden if the size of the graph is very large. In this work, we i…
We develop K$ω$, an open-source linear algebra library for the shifted Krylov subspace methods. The methods solve a set of shifted linear equations $(z_k I-H)x^{(k)}=b\, (k=0,1,2,...)$ for a given matrix $H$ and a vector $b$, simultaneously. The leading order of the operational cost is the same as that for a single equation. The shift invariance of the Krylov subspace is the mathematical foundation of the shifted Krylov subspace methods. Applications in materials science are presented to demonstrate the advantages of the algorithm over the standard Krylov subspace methods such as the Lanczos m…
The present paper is concerned with developing tensor iterative Krylov subspace methods to solve large multi-linear tensor equations. We use the well-known T-product for two tensors to define tensor global Arnoldi and tensor global Gloub-Kahan bidiagonalization algorithms. Furthermore, we illustrate how tensor-based global approaches can be exploited to solve ill-posed problems arising from recovering blurry multichannel (color) images and videos, using the so-called Tikhonov regularization technique, to provide computable approximate regularized solutions. We also review a generalized cross-v…
An accurate residual--time (AccuRT) restarting for computing matrix exponential actions of nonsymmetric matrices by the shift-and-invert (SAI) Krylov subspace method is proposed. The proposed restarting method is an extension of the recently proposed RT (residual--time) restarting and it is designed to avoid a possible accuracy loss in the conventional RT restarting. An expensive part of the SAI Krylov method is solution of linear systems with the shifted matrix. Since the AccuRT algorithm adjusts the shift value, we discuss how the proposed restarting can be implemented with just a single LU~…
