This article is concerned with the numerical solution of subspace optimization problems, consisting of minimizing a smooth functional over the set of orthogonal projectors of fixed rank. Such problems are encountered in particular in electronic structure calculation (Hartree-Fock and Kohn-Sham Density Functional Theory -DFT- models). We compare from a numerical analysis perspective two simple representatives, the damped self-consistent field (SCF) iterations and the gradient descent algorithm, of the two classes of methods competing in the field: SCF and direct minimization methods. We derive asymptotic rates of convergence for these algorithms and analyze their dependence on the spectral gap and other properties of the problem. Our theoretical results are complemented by numerical simulations on a variety of examples, from toy models with tunable parameters to realistic Kohn-Sham computations. We also provide an example of chaotic behavior of the simple SCF iterations for a nonquadratic functional.
In this paper, we propose an overlapping additive Schwarz method for total variation minimization based on a dual formulation. The $O(1/n)$-energy convergence of the proposed method is proven, where $n$ is the number of iterations. In addition, we introduce an interesting convergence property called pseudo-linear convergence of the proposed method; the energy of the proposed method decreases as fast as linearly convergent algorithms until it reaches a particular value. It is shown that such the particular value depends on the overlapping width $δ$, and the proposed method becomes as efficient …
In this paper, we first derive a theoretical basis for spherical conformal parameterizations between a simply connected closed surface $\mathcal{S}$ and a unit sphere $\mathbb{S}^2$ by minimizing the Dirichlet energy on $\overline{\mathbb{C}}$ by stereographic projection. The Dirichlet energy can be rewritten as the sum of the energies associated with the southern and northern hemispheres and can be decreased under an equivalence relation by alternatingly solving the corresponding Laplacian equations. Based on this theoretical foundation, we develop a modified Dirichlet energy minimization wit…
On Parallel and Batch-Cutting Strategies for Norm-Minimization-Based Convex Vector Optimization
King Fahd University of Petroleum & Minerals（法赫德国王石油与矿产大学）；Interdisciplinary Research Center for Smart Mobility and Logistics, King Fahd University of Petroleum & Minerals（法赫德国王石油与矿产大学智能交通与物流交叉研究中心）
The volumetric stretch energy has been widely applied to the computation of volume-/mass-preserving parameterizations of simply connected tetrahedral mesh models. However, this approach still lacks theoretical support. In this paper, we provide the theoretical foundation for volumetric stretch energy minimization (VSEM) to compute volume-/mass-preserving parameterizations. In addition, we develop an associated efficient VSEM algorithm with guaranteed asymptotic R-linear convergence. Furthermore, based on the VSEM algorithm, we propose a projected gradient method for the computation of the volu…
In this paper we consider the composite self-concordant (CSC) minimization problem, which minimizes the sum of a self-concordant function $f$ and a (possibly nonsmooth) proper closed convex function $g$. The CSC minimization is the cornerstone of the path-following interior point methods for solving a broad class of convex optimization problems. It has also found numerous applications in machine learning. The proximal damped Newton (PDN) methods have been well studied in the literature for solving this problem that enjoy a nice iteration complexity. Given that at each iteration these methods t…
We study alternating minimization for matrix completion in the simplest possible setting: completing a rank-one matrix from a revealed subset of the entries. We bound the asymptotic convergence rate by the variational characterization of eigenvalues of a reversible consensus problem. This leads to a polynomial upper bound on the asymptotic rate in terms of number of nodes as well as the largest degree of the graph of revealed entries.
We consider two modifications of the Arrow-Hurwicz (AH) iteration for solving the incompressible steady Navier-Stokes equations for the purpose of accelerating the algorithm: grad-div stabilization, and Anderson acceleration. AH is a classical iteration for general saddle point linear systems and it was later extended to Navier-Stokes iterations in the 1970's which has recently come under study again. We apply recently developed ideas for grad-div stabilization and divergence-free finite element methods along with Anderson acceleration of fixed point iterations to AH in order to improve its co…
Recent quasi-optimal error estimates for the finite element approximation of total-variation regularized minimization problems require the existence of a Lipschitz continuous dual solution. We discuss the validity of this condition and devise numerical methods using locally refined meshes that lead to improved convergence rates despite the occurrence of discontinuities. It turns out that nearly linear convergence is possible on suitably constructed meshes.
We consider sequential and parallel decomposition methods for a dual problem of a general total variation minimization problem with applications in several image processing tasks, like image inpainting, estimation of optical flow and reconstruction of missing wavelet coefficients. The convergence of these methods to a solution of the global problem is analysed in a Hilbert space setting and a convergence rate is provided. Thereby, these convergence result hold not only for exact local minimization but also if the subproblems are just solved approximately. As a concrete example of an approximat…
We present a comprehensive convergence analysis for Self-Consistent Field (SCF) iteration to solve a class of nonlinear eigenvalue problems with eigenvector-dependency (NEPv). Using a tangent-angle matrix as an intermediate measure for approximation error, we establish new formulas for two fundamental quantities that optimally characterize the local convergence of the plain SCF: the local contraction factor and the local average contraction factor. In comparison with previously established results, new convergence rate estimates provide much sharper bounds on the convergence speed. As an appli…
A note on alternating minimization algorithms: Bregman frame
National University of Defense Technology（国防科技大学）；The State Key Laboratory for High Performance Computation, National University of Defense Technology（高性能计算国家重点实验室（国防科技大学））
In this paper, the local convergence of Iteratively regularized Landweber iteration method is investigated for solving non-linear inverse problems in Banach spaces. Our analysis mainly relies on the assumption that the inverse mapping satisfies the Hölder stability estimate locally. We consider both noisy as well as non-noisy data in our analysis. Under the a-priori choice of stopping index for noisy data, we show that the iterates remain in a certain ball around exact solution and obtain the convergence rates. The convergence of the Iteratively regularized Landweber iterates to the exact solu…
In this paper, we first extend the finite distortion problem from the bounded domains in $\mathbb{R}^2$ to the closed genus-zero surfaces in $\mathbb{R}^3$ by the stereographic projection. Then we derive a theoretical foundation for spherical equiareal parameterizations between a simply connected closed surface $\mathcal{M}$ and a unit sphere $\mathbb{S}^2$ via minimizing the total area distortion energy on $\overline{\mathbb{C}}$. Provided we determine the minimizer of the total area distortion energy, the minimizer composed with the initial conformal map determines the equiareal map between …
The dynamic iteration method with a restricted additive Schwarz splitting is investigated to co-simulate linear differential algebraic equations system coming from RLC electrical circuit with linear components. We show the pure linear convergence or divergence of the method with respect to the linear operator belonging to the restricted additive Schwarz interface. It allows us to accelerate it toward the true solution with the Aitken's technique for accelerating convergence. This provides a dynamic iteration method less sensitive to the splitting. Numerical examples with convergent and diverge…
The nonlinear eigen-problem $ Ax+F(x)=λx$ is studied where $A$ is an $n\times n$ irreducible Stieltjes matrix. Under certain conditions, this problem has a unique positive solution. We show that, starting from a multiple of the positive eigenvector of $A$, the Newton-like iteration for this problem converges monotonically. Numerical results illustrate the effectiveness of this Newton-like method.
In this paper, we consider the $\ell_0$ minimization problem whose objective function is the sum of $\ell_0$-norm and convex differentiable function. A variable metric type method which combines the PIHT method and the skill in quasi-newton method, named variable metric extrapolation proximal iterative hard thresholding (VMEPIHT) method, is proposed. Then we analyze its convergence, linear convergence rate and superlinear convergence rate under appropriate assumptions. Finally, we conduct numerical experiments on compressive sensing problem and CT image reconstruction problem to confirm VMPIHT…
We estimate convergence rates for fixed-point iterations of a class of nonlinear operators which are partially motivated from solving convex optimization problems. We introduce the notion of the generalized averaged nonexpansive (GAN) operator with a positive exponent, and provide a convergence rate analysis of the fixed-point iteration of the GAN operator. The proposed generalized averaged nonexpansiveness is weaker than the averaged nonexpansiveness while stronger than nonexpansiveness. We show that the fixed-point iteration of a GAN operator with a positive exponent converges to its fixed-p…
The discrete-time algebraic Riccati equation (DARE) have extensive applications in optimal control problems. We provide new theoretical supports to the stability properties of solutions to the DARE and reduce the convergence conditions under which the accelerated fixed-point iteration (AFPI) can be applied to compute the numerical solutions of DARE. In particular, we verify that the convergence of AFPI is R-superlinear when the spectral radius of the closed-loop matrix is greater than 1, which is shown by mild assumption and only using primary matrix theories. Numerical examples are shown to i…
Fast convergent, accurate, computationally efficient, parallelizable, and robust matrix inversion and parameter estimation algorithms are required in many time-critical and accuracy-critical applications such as system identification, signal and image processing, network and big data analysis, machine learning and in many others. This paper introduces new composite power series expansion with optionally chosen rates (which can be calculated simultaneously on parallel units with different computational capacities) for further convergence rate improvement of high order Newton-Schulz iteration. N…
We consider a model of energy minimization arising in the study of the mechanical behavior caused by cell contraction within a fibrous biological medium. The macroscopic model is based on the theory of non rank-one convex nonlinear elasticity for phase transitions. We study appropriate numerical approximations based on the discontinuous Galerkin treatment of higher gradients and used succesfully in numerical simulations of experiments. We show that the discrete minimizers converge in the limit to minimizers of the continuous problem. This is achieved by employing the theory of $Γ$-convergence …
