With a greedy strategy to construct control index set of coordinates firstly and then choosing the corresponding column submatrix in each iteration, we present a greedy block Gauss-Seidel (GBGS) method for solving large linear least squares problem. Theoretical analysis demonstrates that the convergence factor of the GBGS method can be much smaller than that of the greedy randomized coordinate descent (GRCD) method proposed recently in the literature. On the basis of the GBGS method, we further present a pseudoinverse-free greedy block Gauss-Seidel method, which doesn't need to calculate the Moore-Penrose pseudoinverse of the column submatrix in each iteration any more and hence can be achieved greater acceleration. Moreover, this method can also be used for distributed implementations. Numerical experiments show that, for the same accuracy, our methods can far outperform the GRCD method in terms of the iteration number and computing time.
We present a novel greedy Gauss-Seidel method for solving large linear least squares problem. This method improves the greedy randomized coordinate descent (GRCD) method proposed recently by Bai and Wu [Bai ZZ, and Wu WT. On greedy randomized coordinate descent methods for solving large linear least-squares problems. Numer Linear Algebra Appl. 2019;26(4):1--15], which in turn improves the popular randomized Gauss-Seidel method. Convergence analysis of the new method is provided. Numerical experiments show that, for the same accuracy, our method outperforms the GRCD method in term of the comput…
Refined upper bounds for the convergence of the randomized extended Kaczmarz and Gauss-Seidel algorithms
School of Mathematical Sciences and Fujian Provincial Key Laboratory of Mathematical Modeling and High-Performance Scientific Computing, Xiamen University（厦门大学数学科学学院、福建省数学建模与高性能科学计算重点实验室）
We propose a simple doubly stochastic block Gauss--Seidel algorithm for solving linear systems of equations. By varying the row partition parameter and the column partition parameter of the coefficient matrix, we recover the Landweber algorithm, the randomized Kaczmarz algorithm, the randomized Gauss--Seidel algorithm, and the doubly stochastic Gauss--Seidel algorithm. For general (consistent or inconsistent) linear systems, we show the exponential convergence of the {\it norms of the expected iterates} via exact formulas. For consistent linear systems, we prove the exponential convergence of …
In this paper, a Gauss-Seidel method with oblique direction (GSO) is proposed for finding the least-squares solution to a system of linear equations, where the coefficient matrix may be full rank or rank deficient and the system is overdetermined or underdetermined. Through this method, the number of iteration steps and running time can be reduced to a greater extent to find the least-squares solution, especially when the columns of matrix A are close to linear correlation. It is theoretically proved that GSO method converges to the least-squares solution. At the same time, a randomized versio…
Several applications in optimization, image, and signal processing deal with data that belong to the Stiefel manifold St(n,p), that is, the set of n-by-p matrices with orthonormal columns. Some applications, like the Riemannian center of mass, require evaluating the Riemannian distance between two arbitrary points on St(n,p). This can be done by explicitly constructing the geodesic connecting these two points. An existing method for finding geodesics is the leapfrog algorithm of J. L. Noakes. This algorithm is related to the Gauss-Seidel method, a classical iterative method for solving a linea…
The randomized Gauss--Seidel method and its extension have attracted much attention recently and their convergence rates have been considered extensively. However, the convergence rates are usually determined by upper bounds, which cannot fully reflect the actual convergence. In this paper, we make a detailed analysis of their convergence behaviors. The analysis shows that the larger the singular value of $A$ is, the faster the error decays in the corresponding singular vector space, and the convergence directions are mainly driven by the large singular values at the beginning, then gradually …
In this paper, we present two improved Gauss-Seidel projection methods with unconditional stability. The first method updates the gyromagnetic term and the damping term simultaneously and follows by a projection step. The second method introduces two sets of approximate solutions, where we update the gyromagnetic term and the damping term simultaneously for one set of approximate solutions and apply the projection step to the other set of approximate solutions in an alternating manner. Compared to the original Gauss-Seidel projection method which has to solve heat equations $7$ times at each t…
Aitken extrapolation normally applied to convergent fixed point iteration is extended to extrapolate the solution of a divergent iteration. In addition, higher order Aitken extrapolation is introduced that enables successive decomposition of high Eigen values of the iteration matrix to enable convergence. While extrapolation of a convergent fixed point iteration using a geometric series sum is a known form of Aitken acceleration, it is shown in this paper that the same formula can be used to estimate the solution of sets of linear equations from diverging Gauss Seidel iterations. In both conve…
Large-scale linear complementarity problems (LCPs) are repeatedly solved in interactive rigid-body simulations. The projected Gauss-Seidel method is often employed for LCPs, since it has advantages in computation time, numerical robustness, and memory use. Zheng and Yin (2013) proposed modulus-based matrix splitting iteration methods and showed their effectiveness for large problems, but a simple application of their approach to large-scale LCPs in interactive rigid-body simulations is not effective since such a simple application demands large matrix multiplications. In this paper, we propose…
The GMRES algorithm of Saad and Schultz (1986) is an iterative method for approximately solving linear systems $A{\bf x}={\bf b}$, with initial guess ${\bf x}_0$ and residual ${\bf r}_0 = {\bf b} - A{\bf x}_0$. The algorithm employs the Arnoldi process to generate the Krylov basis vectors (the columns of $V_k$). It is well known that this process can be viewed as a $QR$ factorization of the matrix $B_k = [\: {\bf r}_0, AV_k\:]$ at each iteration. Despite an ${O}(ε)κ(B_k)$ loss of orthogonality, for unit roundoff $ε$ and condition number $κ$, the modified Gram-Schmidt formulation was shown to b…
Exploiting Task-Based Parallelism for the Red-Black Gauss-Seidel Method on 2D Grids
KTH Royal Institute of Technology（皇家理工学院）；Forschungszentrum Jülich GmbH（于利希研究中心有限公司）；Bergische Universität Wuppertal（伍珀塔尔大学）；University of Groningen（格罗宁根大学）
Some variants of the (block) Gauss--Seidel iteration for the solution of linear systems with $M$-matrices in (block) Hessenberg form are discussed. Comparison results for the asymptotic convergence rate of some regular splittings are derived: in particular, we prove that for a lower-Hessenberg M-matrix $ρ(P_{GS})\geq ρ(P_S)\geq ρ(P_{AGS})$, where $P_{GS}, P_S, P_{AGS}$ are the iteration matrices of the Gauss--Seidel, staircase, and anti-Gauss--Seidel method. This is a result that does not seem to follow from classical comparison results, as these splittings are not directly comparable. It is s…
We extend results known for the randomized Gauss-Seidel and the Gauss-Southwell methods for the case of a Hermitian and positive definite matrix to certain classes of non-Hermitian matrices. We obtain convergence results for a whole range of parameters describing the probabilities in the randomized method or the greedy choice strategy in the Gauss-Southwell-type methods. We identify those choices which make our convergence bounds best possible. Our main tool is to use weighted l1-norms to measure the residuals. A major result is that the best convergence bounds that we obtain for the expected …
The fully implicit method is the most commonly used approach to solve black-oil problems in reservoir simulation. The method requires repeated linearization of large nonlinear systems and produces ill-condi\-tioned linear systems. We present a strategy to reduce computational time that relies on two key ideas: (\textit{i}) a sequential formulation that decouples flow and transport into separate subproblems, and (\textit{ii}) a highly efficient Gauss--Seidel solver for the transport problems. This solver uses intercell fluxes to reorder the grid cells according to their upstream neighbors, and …
Gauss-Seidel (GS) relaxation is often employed as a preconditioner for a Krylov solver or as a smoother for Algebraic Multigrid (AMG). However, the requisite sparse triangular solve is difficult to parallelize on many-core architectures such as graphics processing units (GPUs). In the present study, the performance of the traditional GS relaxation based on a triangular solve is compared with two-stage variants, replacing the direct triangular solve with a fixed number of inner Jacobi-Richardson (JR) iterations. When a small number of inner iterations is sufficient to maintain the Krylov conver…
Magnetization dynamics in magnetic materials is often modeled by the Landau-Lifshitz equation, which is solved numerically in general. In micromagnetic simulations, the computational cost relies heavily on the time-marching scheme and the evaluation of stray field. Explicit marching schemes are efficient but suffer from severe stability constraints, while nonlinear systems of equations have to be solved in implicit schemes though they are unconditionally stable. A better compromise between stability and efficiency is the semi-implicit scheme, such as the Gauss-Seidel projection method (GSPM) a…
A Dual Symmetric Gauss-Seidel Alternating Direction Method of Multipliers for Hyperspectral Sparse Unmixing
School of Information Science and technology, and the Provincial Key Lab of Information Coding and Trans- mission, Southwest Jiaotong University（信息科学与技术学院，信息编码与传输省重点实验室，西南交通大学）；School of Mathematics, Southwest Jiaotong University（数学学院，西南交通大学）；School of Computer and Computing Science, Zhejiang University City College（计算机与计算科学学院，浙江大学城市学院）
Analysis of Block Jacobi/Gauss-Seidel and additive/multiplicative Schwarz preconditioning through the theory of GLT sequences, with applications to domain decomposition discretizations
University of Rome Tor Vergata（罗马托尔维加塔大学）；University Mohammed VI Polytechnic（穆罕默德六世polytechnic大学）；University of Insubria（因苏布里亚大学）；Uppsala University（乌普萨拉大学）
Greedy Gauss-Newton algorithm for finding sparse solutions to nonlinear underdetermined systems of equations
School of Science and Technology, Örebro University（欧勒布鲁大学理学院与技术学院）；Department of Mathematical Sciences and Technology, Norwegian University of Life Sciences（挪威生命科学大学数学科学与技术系）
A new preconditioner based on a block $LDU$ factorization with algebraic multigrid subsolves for scalability is introduced for the large, structured systems appearing in implicit Runge-Kutta time integration of parabolic partial differential equations. This preconditioner is compared in condition number and eigenvalue distribution, and in numerical experiments with others in the literature: block Jacobi, block Gauss-Seidel, and the optimized block Gauss-Seidel method of Staff, Mardal, and Nilssen [{\em Modeling, Identification and Control}, 27 (2006), pp. 109-123]. Experiments are run on two t…
