Preconditioned eigenvalue solvers (eigensolvers) are gaining popularity, but their convergence theory remains sparse and complex. We consider the simplest preconditioned eigensolver--the gradient iterative method with a fixed step size--for symmetric generalized eigenvalue problems, where we use the gradient of the Rayleigh quotient as an optimization direction. A sharp convergence rate bound for this method has been obtained in 2001--2003. It still remains the only known such bound for any of the methods in this class. While the bound is short and simple, its proof is not. We extend the bound to Hermitian matrices in the complex space and present a new self-contained and significantly shorter proof using novel geometric ideas.
Many popular eigensolvers for large and sparse Hermitian matrices or matrix pairs can be interpreted as accelerated block preconditioned gradient (BPG) iterations in order to analyze their convergence behavior by composing known estimates. An important feature of BPG is the cluster robustness, i.e., reasonable performance for computing clustered eigenvalues is ensured by a sufficiently large block size. This feature can easily be explained for exact-inverse (exact shift-inverse) preconditioning by adapting classical estimates on nonpreconditioned eigensolvers, whereas the existing results for …
Gradient-type iterative methods for solving Hermitian eigenvalue problems can be accelerated by using preconditioning and deflation techniques. A preconditioned steepest descent iteration with implicit deflation (PSD-id) is one of such methods. The convergence behavior of the PSD-id is recently investigated based on the pioneering work of Samokish on the preconditioned steepest descent method (PSD). The resulting non-asymptotic estimates indicate a superlinear convergence of the PSD-id under strong assumptions on the initial guess. The present paper utilizes an alternative convergence analysis…
Convergence theory for preconditioned eigenvalue solvers in a nutshell
University Colorado Denver（科罗拉多大学丹佛分校）；Mitsubishi Electric Research Laboratories（三菱电机研究实验室）；Universität Rostock（罗斯托克大学）；STFC Rutherford Appleton Laboratory（STFC鲁瑟福阿普尔顿实验室）
This paper introduces bootstrap multigrid methods for solving eigenvalue problems arising from the discretization of partial differential equations. Inspired by the full bootstrap algebraic multigrid (BAMG) setup algorithm that includes an AMG eigensolver, it is illustrated how the algorithm can be simplified for the case of a discretized partial differential equation (PDE), thereby developing a bootstrap geometric multigrid (BMG) approach. We illustrate numerically the efficacy of the BMG method for: (1) recovering eigenvalues having large multiplicity, (2) computing interior eigenvalues, and…
We develop a theoretical foundation for the application of Nesterov's accelerated gradient descent method (AGD) to the approximation of solutions of a wide class of partial differential equations (PDEs). This is achieved by proving the existence of an invariant set and exponential convergence rates when its preconditioned version (PAGD) is applied to minimize locally Lipschitz smooth, strongly convex objective functionals. We introduce a second-order ordinary differential equation (ODE) with a preconditioner built-in and show that PAGD is an explicit time-discretization of this ODE, which requ…
Primal-dual interior-point multigrid method for topology optimization
University of Birmingham（伯明翰大学）；Institute of Information Theory and Automation, Academy of Sciences of the Czech Republic（捷克科学院信息论与自动化研究所）；University of Kirkuk（基尔库克大学）
In this work, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Sto…
Generalized preconditioned conjugate gradients for adaptive FEM with optimal complexity
Sorbonne Université, Université Paris Cité, CNRS, Laboratoire Jacques-Louis Lions（索邦大学，巴黎 Cité 大学，法国国家科学研究中心，雅克-路易·利翁斯实验室）；TU Wien, Institute of Analysis and Scientific Computing（维也纳工业大学，分析与科学计算研究所）
Finding fast yet accurate numerical solutions to the Helmholtz equation remains a challenging task. The pollution error (i.e. the discrepancy between the numerical and analytical wave number k) requires the mesh resolution to be kept fine enough to obtain accurate solutions. A recent study showed that the use of Isogeometric Analysis (IgA) for the spatial discretization significantly reduces the pollution error. However, solving the resulting linear systems by means of a direct solver remains computationally expensive when large wave numbers or multiple dimensions are considered. An alternativ…
