Deterministic and randomized, row-action and column-action linear solvers have become increasingly popular owing to their simplicity, low computational and memory complexities, and ease of composition with other techniques. Moreover, in order to achieve high-performance, such solvers must often be adapted to the given problem structure and to the hardware platform on which the problem will be solved. Unfortunately, determining whether such adapted solvers will converge to a solution has required equally unique analyses. As a result, adapted, reliable solvers are slow to be developed and deployed. In this work, we provide a general set of assumptions under which such adapted solvers are guaranteed to converge with probability one, and provide worst case rates of convergence. As a result, we can provide practitioners with guidance on how to design highly adapted, randomized or deterministic, row-action or column-action linear solvers that are also guaranteed to converge.
Randomized orthogonal projection methods (ROPMs) can be used to speed up the computation of Krylov subspace methods in various contexts. Through a theoretical and numerical investigation, we establish that these methods produce quasi-optimal approximations over the Krylov subspace. Our numerical experiments outline the convergence of ROPMs for all matrices in our test set, with occasional spikes, but overall with a convergence rate similar to that of standard OPMs.
Randomized iterative algorithms have attracted much attention in recent years because they can approximately solve large-scale linear systems of equations without accessing the entire coefficient matrix. In this paper, we propose two novel pseudoinverse-free randomized block iterative algorithms for solving consistent and inconsistent linear systems. The proposed algorithms require two user-defined random matrices: one for row sampling and the other for column sampling. We can recover the well-known doubly stochastic Gauss--Seidel, randomized Kaczmarz, randomized coordinate descent, and random…
The indefinite least squares (ILS) problem is a generalization of the famous linear least squares problem. It minimizes an indefinite quadratic form with respect to a signature matrix. For this problem, we first propose an impressively simple and effective splitting (SP) method according to its own structure and prove that it converges 'unconditionally' for any initial value. Further, to avoid implementing some matrix multiplications and calculating the inverse of large matrix and considering the acceleration and efficiency of the randomized strategy, we develop two randomized iterative method…
Randomized linear system solvers have become popular as they have the potential to reduce floating point complexity while still achieving desirable convergence rates. One particularly promising class of methods, random sketching solvers, has achieved the best known computational complexity bounds in theory, but is blunted by two practical considerations: there is no clear way of choosing the size of the sketching matrix apriori; and there is a nontrivial storage cost of the sketched system. In this work, we make progress towards addressing these issues by implicitly generating the sketched sys…
We present a fast iterative solver for scattering problems in 2D, where a penetrable object with compact support is considered. By representing the scattered field as a volume potential in terms of the Green's function, we arrive at the Lippmann-Schwinger equation in integral form, which is then discretized using an appropriate quadrature technique. The discretized linear system is then solved using an iterative solver accelerated by Directional Algebraic Fast Multipole Method (DAFMM). The DAFMM presented here relies on the directional admissibility condition of the 2D Helmholtz kernel. And th…
We study computational complexity aspects for Finite Element formulations considering hypercubic space--time full and time--marching discretization schemes for $h$--refined grids towards singularities. We perform a relatively comprehensive study of comparing the computational time via time complexities of direct and iterative solvers. We focus on the space-time formulation with refined computational grids and on the corresponding time slabs, namely, computational grids obtained by taking the "cross-sections" of the refined space-time mesh. We compare the computational complexity of the space-t…
This paper presents the results of a preliminary experimental investigation of the performance of a stationary iterative method based on a block staircase splitting for solving singular systems of linear equations arising in Markov chain modelling. From the experiments presented, we can deduce that the method is well suited for solving block banded or more generally localized systems in a parallel computing environment. The parallel implementation has been benchmarked using several Markovian models.
We consider the goal-oriented error estimates for a linearized iterative solver for nonlinear partial differential equations. For the adjoint problem and iterative solver we consider, instead of the differentiation of the primal problem, a suitable linearization which guarantees the adjoint consistency of the numerical scheme. We derive error estimates and develop an efficient adaptive algorithm which balances the errors arising from the discretization and use of iterative solvers. Several numerical examples demonstrate the efficiency of this algorithm.
This paper introduces a discretization-accurate stopping criterion of symmetric iterative methods for solving systems of algebraic equations resulting from the finite element approximation. The stopping criterion consists of the evaluations of the discretization and the algebraic error estimators, that are based on the respective duality error estimator and the difference of two consecutive iterates. Iterations are terminated when the algebraic estimator is of the same magnitude as the discretization estimator. Numerical results for multigrid $V(1,1)$-cycle and symmetric Gauss-Seidel iterative…
In this paper, we develop a (preconditioned) GMRES solver based on integer arithmetic, and introduce an iterative refinement framework for the solver. We describe the data format for the coefficient matrix and vectors for the solver that is based on integer or fixed-point numbers. To avoid overflow in calculations, we introduce initial scaling and logical shifts (adjustments) of operands in arithmetic operations. We present the approach for operand shifts, considering the characteristics of the GMRES algorithm. Numerical tests demonstrate that the integer arithmetic-based solver with iterative…
Regularization Properties of the Krylov Iterative Solvers CGME and LSMR For Linear Discrete Ill-Posed Problems with an Application to Truncated Randomized SVDs
The discovery of fast numerical solvers prompted a clear and rapid shift towards iterative techniques in many applications, especially in computational mechanics, due to the increased necessity for solving very large linear systems. Most numerical solvers are highly dependent on the problem geometry and discretization, facing issues when any of these properties change. The newly developed Hybrid Iterative Numerical Transferable Solver (HINTS) combines a standard solver with a neural operator to achieve better performance, focusing on a single geometry at a time. In this work, we explore the "T…
We present a scalable and efficient iterative solver for high-order hybridized discontinuous Galerkin (HDG) discretizations of hyperbolic partial differential equations. It is an interplay between domain decomposition methods and HDG discretizations. In particular, the method is a fixed-point approach that requires only independent element-by-element local solves in each iteration. As such, it is well-suited for current and future computing systems with massive concurrencies. We rigorously show that the proposed method is exponentially convergent in the number of iterations for transport and l…
In this thesis, the numerical solution of three different classes of problems have been studied. Specifically, new techniques have been proposed and their theoretical analysis has been performed, accompanied by a wide set of numerical experiments, for investigating further and comparing the effectiveness and performance of the presented approach. The first two belong to the research area of numerical linear algebra and concern the spectral analysis and preconditioning for Krylov subspace methods of the coefficient matrix of large structured linear systems. The third concerns a problem from the…
We present and analyze a new iterative solver for implicit discretizations of a simplified Boltzmann-Poisson system. The algorithm builds on recent work that incorporated a sweeping algorithm for the Vlasov-Poisson equations as part of nested inner-outer iterative solvers for the Boltzmann-Poisson equations. The new method eliminates the need for nesting and requires only one transport sweep per iteration. It arises as a new fixed-point formulation of the discretized system which we prove to be contractive for a given electric potential. We also derive an accelerator to improve the convergence…
University of Edinburgh（爱丁堡大学）；King Abdullah University of Science and Technology (KAUST)（沙特国王 Abdullah 科技大学 (KAUST)）；MIPT（莫斯科物理技术学院）
In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic subspace ascent. A common feature of these methods is that in their update rule a certain sub-problem needs to be solved exactly. We relax this requirement by allowing for the sub-problem to be solved inexactly. In particular, we propose and analyze inexact randomized iterative methods for solving three closely related problems: a convex stochastic quadratic optim…
We present a scalability study of Golub-Kahan bidiagonalization for the parallel iterative solution of symmetric indefinite linear systems with a 2x2 block structure. The algorithms have been implemented within the parallel numerical library PETSc. Since a nested inner-outer iteration strategy may be necessary, we investigate different choices for the inner solvers, including parallel sparse direct and multigrid accelerated iterative methods. We show the strong and weak scalability of the Golub-Kahan bidiagonalization based iterative method when applied to a two-dimensional Poiseuille flow and…
This manuscript presents an efficient solver for the linear system that arises from the Hierarchical Poincaré-Steklov (HPS) discretization of three dimensional variable coefficient Helmholtz problems. Previous work on the HPS method has tied it with a direct solver. This work is the first efficient iterative solver for the linear system that results from the HPS discretization. The solution technique utilizes GMRES coupled with a locally homogenized block-Jacobi preconditioner. The local nature of the discretization and preconditioner naturally yield the matrix-free application of the linear s…
Constrained Optimization via Exact Augmented Lagrangian and Randomized Iterative Sketching
University of Chicago（芝加哥大学）；University of California, Berkeley（加州大学伯克利分校）；International Computer Science Institute（国际计算机科学研究所）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
The Kaczmarz method is an iterative algorithm for solving systems of linear equalities and inequalities, that iteratively projects onto these constraints. Recently, Strohmer and Vershynin [J. Fourier Anal. Appl., 15(2):262-278, 2009] gave a non-asymptotic convergence rate analysis for this algorithm, spurring numerous extensions and generalizations of the Kaczmarz method. Rather than the randomized selection rule analyzed in that work, in this paper we instead discuss greedy and approximate greedy selection rules. We show that in some applications the computational costs of greedy and random s…
