The higher-order orthogonality iteration (HOOI) has been popularly used for finding a best low-multilinear-rank approximation of a tensor. However, its iterate sequence convergence is still an open question. In this paper, we first analyze a greedy HOOI, which updates each factor matrix by selecting from the best candidates one that is closest to the current iterate. Assuming the existence of a block-nondegenerate limit point, we establish its global convergence through the so-called Kurdyka-Lojasiewicz (KL) property. In addition, we show that if the starting point is sufficiently close to any block-nondegenerate globally optimal solution, the greedy HOOI produces a sequence convergent to a globally optimal solution. Relating the iterate sequence by the original HOOI to that by the greedy HOOI, we then show that the same convergence results hold for the original HOOI and thus positively address the open question.
The Kaczmarz method is an iterative algorithm for solving systems of linear equalities and inequalities, that iteratively projects onto these constraints. Recently, Strohmer and Vershynin [J. Fourier Anal. Appl., 15(2):262-278, 2009] gave a non-asymptotic convergence rate analysis for this algorithm, spurring numerous extensions and generalizations of the Kaczmarz method. Rather than the randomized selection rule analyzed in that work, in this paper we instead discuss greedy and approximate greedy selection rules. We show that in some applications the computational costs of greedy and random s…
A fundamental class of matrix optimization problems that arise in many areas of science and engineering is that of quadratic optimization with orthogonality constraints. Such problems can be solved using line-search methods on the Stiefel manifold, which are known to converge globally under mild conditions. To determine the convergence rate of these methods, we give an explicit estimate of the exponent in a Lojasiewicz inequality for the (non-convex) set of critical points of the aforementioned class of problems. By combining such an estimate with known arguments, we are able to establish the …
A simple but successful strategy for building a discrete diffusion operator in finite volume schemes of industrial use is to correct the standard two-point flux approximation with a term accounting for the local mesh non-orthogonality. Practical experience with a variety of different mesh typologies, including non-orthogonal tetrahedral, hexahedral and polyhedral meshes, has shown that this discrete diffusion operator is accurate and robust whenever the mesh is not too distorted and sufficiently regular. In this work, we show that this approach can be interpreted as equivalent to introducing a…
Convergence and Optimality of Adaptive Mixed Finite Element Methods
University of California at Irvine（加州大学尔湾分校）；University of California at San Diego（加州大学圣地亚哥分校）；Peking University（北京大学）；Pennsylvania State University（宾夕法尼亚州立大学）
We consider two modifications of the Arrow-Hurwicz (AH) iteration for solving the incompressible steady Navier-Stokes equations for the purpose of accelerating the algorithm: grad-div stabilization, and Anderson acceleration. AH is a classical iteration for general saddle point linear systems and it was later extended to Navier-Stokes iterations in the 1970's which has recently come under study again. We apply recently developed ideas for grad-div stabilization and divergence-free finite element methods along with Anderson acceleration of fixed point iterations to AH in order to improve its co…
We present a comprehensive convergence analysis for Self-Consistent Field (SCF) iteration to solve a class of nonlinear eigenvalue problems with eigenvector-dependency (NEPv). Using a tangent-angle matrix as an intermediate measure for approximation error, we establish new formulas for two fundamental quantities that optimally characterize the local convergence of the plain SCF: the local contraction factor and the local average contraction factor. In comparison with previously established results, new convergence rate estimates provide much sharper bounds on the convergence speed. As an appli…
In this paper, the local convergence of Iteratively regularized Landweber iteration method is investigated for solving non-linear inverse problems in Banach spaces. Our analysis mainly relies on the assumption that the inverse mapping satisfies the Hölder stability estimate locally. We consider both noisy as well as non-noisy data in our analysis. Under the a-priori choice of stopping index for noisy data, we show that the iterates remain in a certain ball around exact solution and obtain the convergence rates. The convergence of the Iteratively regularized Landweber iterates to the exact solu…
The dynamic iteration method with a restricted additive Schwarz splitting is investigated to co-simulate linear differential algebraic equations system coming from RLC electrical circuit with linear components. We show the pure linear convergence or divergence of the method with respect to the linear operator belonging to the restricted additive Schwarz interface. It allows us to accelerate it toward the true solution with the Aitken's technique for accelerating convergence. This provides a dynamic iteration method less sensitive to the splitting. Numerical examples with convergent and diverge…
This paper considers an enhancement of the classical iterated penalty Picard (IPP) method for the incompressible Navier-Stokes equations, where we restrict our attention to $O(1)$ penalty parameter, and Anderson acceleration (AA) is used to significantly improve its convergence properties. After showing the fixed point operator associated with the IPP iteration is Lipschitz continuous and Lipschitz continuously (Frechet) differentiable, we apply a recently developed general theory for AA to conclude that IPP enhanced with AA improves its linear convergence rate by the gain factor associated wi…
In Part I of this paper, we introduced a two dimensional eigenvalue problem (2DEVP) of a matrix pair and investigated its fundamental theory such as existence, variational characterization and number of 2D-eigenvalues. In Part II, we proposed a Rayleigh quotient iteration (RQI)-like algorithm (2DRQI) for computing a 2D-eigentriplet of the 2DEVP near a prescribed point, and discussed applications of 2DEVP and 2DRQI for solving the minimax problem of Rayleigh quotients, and computing the distance to instability. In this third part, we present convergence analysis of the 2DRQI. We show that under…
We propose a new modification of Newton iteration for finding some nonnegative Z-eigenpairs of a nonnegative tensor. The method has local quadratic convergence to a nonnegative eigenpair of a nonnegative tensor, under the usual assumption guaranteeing the local quadratic convergence of the original Newton iteration.
We introduce an iterative method named GPMR for solving 2x2 block unsymmetric linear systems. GPMR is based on a new process that reduces simultaneously two rectangular matrices to upper Hessenberg form and that is closely related to the block-Arnoldi process. GPMR is tantamount to Block-GMRES with two right-hand sides in which the two approximate solutions are summed at each iteration, but requires less storage and work per iteration. We compare the performance of GPMR with GMRES and Block-GMRES on linear systems from the SuiteSparse Matrix Collection. In our experiments, GPMR terminates sign…
We give a self-contained randomized algorithm based on shifted inverse iteration which provably computes the eigenvalues of an arbitrary matrix $M\in\mathbb{C}^{n\times n}$ up to backward error $δ\|M\|$ in $O(n^4+n^3\log^2(n/δ)+\log(n/δ)^2\log\log(n/δ))$ floating point operations using $O(\log^2(n/δ))$ bits of precision. While the $O(n^4)$ complexity is prohibitive for large matrices, the algorithm is simple and may be useful for provably computing the eigenvalues of small matrices using controlled precision, in particular for computing Ritz values in shifted QR algorithms as in (Banks, Garza-…
The fixed-stress splitting scheme is a popular method for iteratively solving the Biot equations. The method successively solves the flow and mechanic subproblems while adding a stabilizing term to the flow equation, which includes a parameter that can be chosen freely. However, the convergence properties of the scheme depend significantly on this parameter and choosing it carelessly might lead to a very slow, or even diverging, method. In this paper, we present a way to exploit the matrix structure arizing from discretizing the equations in the regime of impermeable porous media in order to o…
The nonlinear eigen-problem $ Ax+F(x)=λx$ is studied where $A$ is an $n\times n$ irreducible Stieltjes matrix. Under certain conditions, this problem has a unique positive solution. We show that, starting from a multiple of the positive eigenvector of $A$, the Newton-like iteration for this problem converges monotonically. Numerical results illustrate the effectiveness of this Newton-like method.
For solving a class of block two-by-two real linear system, a new single-step iteration method based on triangular splitting scheme is proposed in this paper. Then the convergence properties of this method are carefully investigated. In addition, we determine its optimal iteration parameters and give the corresponding optimal convergence factor. It is worth mentioning that the SSTS iteration method is robust and superior to SBTS and PSBTS iteration methods under suitable conditions. Finally, some numerical experiments are carried out to validate the theoretical results and evaluate this new me…
Li, Chen, Tai & E. (J. Machine Learning Research, 2018) have proposed a regularization of the forward-backward sweep iteration for solving the Pontryagin maximum principle in optimal control problems. The authors prove the global convergence of the iteration in the continuous time case. In this article we show that their proof can be extended to the case of numerical discretization by symplectic Runge-Kutta pairs. We demonstrate the convergence with a simple numerical experiment.
We estimate convergence rates for fixed-point iterations of a class of nonlinear operators which are partially motivated from solving convex optimization problems. We introduce the notion of the generalized averaged nonexpansive (GAN) operator with a positive exponent, and provide a convergence rate analysis of the fixed-point iteration of the GAN operator. The proposed generalized averaged nonexpansiveness is weaker than the averaged nonexpansiveness while stronger than nonexpansiveness. We show that the fixed-point iteration of a GAN operator with a positive exponent converges to its fixed-p…
The discrete-time algebraic Riccati equation (DARE) have extensive applications in optimal control problems. We provide new theoretical supports to the stability properties of solutions to the DARE and reduce the convergence conditions under which the accelerated fixed-point iteration (AFPI) can be applied to compute the numerical solutions of DARE. In particular, we verify that the convergence of AFPI is R-superlinear when the spectral radius of the closed-loop matrix is greater than 1, which is shown by mild assumption and only using primary matrix theories. Numerical examples are shown to i…
Fast convergent, accurate, computationally efficient, parallelizable, and robust matrix inversion and parameter estimation algorithms are required in many time-critical and accuracy-critical applications such as system identification, signal and image processing, network and big data analysis, machine learning and in many others. This paper introduces new composite power series expansion with optionally chosen rates (which can be calculated simultaneously on parallel units with different computational capacities) for further convergence rate improvement of high order Newton-Schulz iteration. N…
This article is concerned with the numerical solution of subspace optimization problems, consisting of minimizing a smooth functional over the set of orthogonal projectors of fixed rank. Such problems are encountered in particular in electronic structure calculation (Hartree-Fock and Kohn-Sham Density Functional Theory -DFT- models). We compare from a numerical analysis perspective two simple representatives, the damped self-consistent field (SCF) iterations and the gradient descent algorithm, of the two classes of methods competing in the field: SCF and direct minimization methods. We derive …
For a linear complementarity problem, we present a relaxaiton accelerated two-sweep matrix splitting iteration method. The convergence analysis illustrates that the proposed method converges to the exact solution of the linear complementarity problem when the system matrix is an $H_+$-matrix and the convergence conditions are given. Numerical experiments show that the proposed method is more efficient than the existing ones.
