In this paper, a new analytic method with a convergence-control parameter $c$ is first proposed. The parameter $c$ is used to adjust and control the convergence region and rate of the resulting series solution. It turns out that the convergence region and rate can be greatly enlarged by choosing a proper value of $c$. Furthermore, a numerical approach for finding the optimal value of the convergence-control parameter is given. At the same time, it is found that the traditional Adomian decomposition method is only a special case of the new method. The effectiveness and applicability of the new technique are demonstrated by several physical models including nonlinear heat transfer problems, nano-electromechanical systems, diffusion and dissipation phenomena, and dispersive waves. Moreover, the ideas proposed in this paper may offer us possibilities to greatly improve current analytic and numerical techniques.
This paper studies the performance Newton's iteration applied with Anderson acceleration for solving the incompressible steady Navier-Stokes equations. We manifest that this method converges superlinearly with a good initial guess, and moreover, a large Anderson depth decelerates the convergence speed comparing to a small Anderson depth. We observe that the numerical tests confirm these analytical convergence results, and in addition, Anderson acceleration sometimes enlarges the domain of convergence for Newton's method.
We present a comprehensive convergence analysis for Self-Consistent Field (SCF) iteration to solve a class of nonlinear eigenvalue problems with eigenvector-dependency (NEPv). Using a tangent-angle matrix as an intermediate measure for approximation error, we establish new formulas for two fundamental quantities that optimally characterize the local convergence of the plain SCF: the local contraction factor and the local average contraction factor. In comparison with previously established results, new convergence rate estimates provide much sharper bounds on the convergence speed. As an appli…
Convergence Analysis for A Class of Iterative Methods for Solving Saddle Point Systems
University of California at Irvine（加州大学尔湾分校）；Beijing Institute for Scientific and Engineering Computing, Beijing University of Technology（北京工业大学北京科学计算与工程研究所）；School of Mathematical Sciences, University of Electronic Science and Technology of China（电子科技大学数学科学学院）
We extend results known for the randomized Gauss-Seidel and the Gauss-Southwell methods for the case of a Hermitian and positive definite matrix to certain classes of non-Hermitian matrices. We obtain convergence results for a whole range of parameters describing the probabilities in the randomized method or the greedy choice strategy in the Gauss-Southwell-type methods. We identify those choices which make our convergence bounds best possible. Our main tool is to use weighted l1-norms to measure the residuals. A major result is that the best convergence bounds that we obtain for the expected …
We present an error analysis for the discontinuous Galerkin method applied to the discrete-ordinate discretization of the steady-state radiative transfer equation. Under some mild assumptions, we show that the DG method converges uniformly with respect to a scaling parameter $\varepsilon$ which characterizes the strength of scattering in the system. However, the rate is not optimal and can be polluted by the presence of boundary layers. In one-dimensional slab geometries, we demonstrate optimal convergence when boundary layers are not present and analyze a simple strategy for balance interior …
The discrete-time algebraic Riccati equation (DARE) have extensive applications in optimal control problems. We provide new theoretical supports to the stability properties of solutions to the DARE and reduce the convergence conditions under which the accelerated fixed-point iteration (AFPI) can be applied to compute the numerical solutions of DARE. In particular, we verify that the convergence of AFPI is R-superlinear when the spectral radius of the closed-loop matrix is greater than 1, which is shown by mild assumption and only using primary matrix theories. Numerical examples are shown to i…
In this work, we consider a boundary value problem for nonlinear triharmonic equation. Due to the reduction of nonlinear boundary value problems to operator equation for nonlinear terms we establish the existence, uniqueness and positivity of solution. More importantly, we design an iterative method at both continuous and discrete level for numerical solution of the problem. An analysis of actual total error of the obtained discrete solution is made. Some examples demonstrate the applicability of the theoretical results on qualitative aspects and the efficiency of the iterative method.
In this paper we consider a fully third order nonlinear boundary value problem which is of great interest of many researchers. First we establish the existence, uniqueness of solution. Next, we propose simple iterative methods on both continuous and discrete levels. We prove that the discrete methods are of second order and third accuracy due to the use of appropriate formulas for numerical integration and obtain estimate for total error. Some examples demonstrate the validity of the obtained theoretical results and the efficiency of the iterative method.
We investigate an iterative mean value method for the inverse (and highly ill-posed) problem of solving the heat equation backwards in time. Semi-group theory is used to rewrite the solution of the inverse problem as the solution of a fixed point equation for an affine operator, with linear part satisfying special functional analytical properties. We give a convergence proof for the method and obtain convergence rates for the residual. Convergence rates for the iterates are also obtained under the so called source conditions.
In this article, we develop and analyze a finite element method with the first family Nédélec elements of the lowest degree for solving a Maxwell interface problem modeled by a $\mathbf{H}(\text{curl})$-elliptic equation on unfitted meshes. To capture the jump conditions optimally, we construct and use $\mathbf{H}(\text{curl})$ immersed finite element (IFE) functions on interface elements while keep using the standard Nédélec functions on all the non-interface elements. We establish a few important properties for the IFE functions including the unisolvence according to the edge degrees of free…
Free boundary problems deal with systems of partial differential equations, where the domain boundaries are apriori unknown. Due to this special characteristic, it is challenging to solve free boundary problems either theoretically or numerically. In this paper, we develop a novel approach for solving a modified Hele-Shaw problem based on neural network discretization. The existence of the numerical solution with this discretization is established theoretically. We also numerically verify this approach by computing the symmetry-breaking solutions which are guided by the bifurcation analysis ne…
In this paper, for solving large-scale nonlinear equations we propose a nonlinear sampling Kaczmarz-Motzkin (NSKM) method. Based on the local tangential cone condition and the Jensen's inequality, we prove convergence of our method with two different assumptions. Then, for solving nonlinear equations with the convex constraints we present two variants of the NSKM method: the projected sampling Kaczmarz-Motzkin (PSKM) method and the accelerated projected sampling Kaczmarz-Motzkin (APSKM) method. With the use of the nonexpansive property of the projection and the convergence of the NSKM method, …
A randomized Kaczmarz method was recently proposed for phase retrieval, which has been shown numerically to exhibit empirical performance over other state-of-the-art phase retrieval algorithms both in terms of the sampling complexity and in terms of computation time. While the rate of convergence has been studied well in the real case where the signals and measurement vectors are all real-valued, there is no guarantee for the convergence in the complex case. In fact, the linear convergence of the randomized Kaczmarz method for phase retrieval in the complex setting is left as a conjecture by T…
Solving large-scale systems of nonlinear equations/inequalities is a fundamental problem in computing and optimization. In this paper, we propose a generic successive projection (SP) framework for this problem. The SP sequentially projects the current iterate onto the constraint set corresponding to each nonlinear (in)equality. It extends von Neumann's alternating projection for finding a point in the intersection of two linear subspaces, Bregman's method for finding a common point of convex sets and the Kaczmarz method for solving systems of linear equations to the more general case of multip…
Two neural-network-based numerical schemes are proposed to solve the classical obstacle problems. The schemes are based on the universal approximation property of neural networks, and the cost functions are taken as the energy minimization of the obstacle problems. We rigorously prove the convergence of the two schemes and derive the convergence rates with the number of neurons $N$. In the simulations, we use two example problems (1-D & 2-D) to verify the convergence rate of the methods and the quality of the results.
Deep learning based reconstruction methods deliver outstanding results for solving inverse problems and are therefore becoming increasingly important. A recently invented class of learning-based reconstruction methods is the so-called NETT (for Network Tikhonov Regularization), which contains a trained neural network as regularizer in generalized Tikhonov regularization. The existing analysis of NETT considers fixed operator and fixed regularizer and analyzes the convergence as the noise level in the data approaches zero. In this paper, we extend the frameworks and analysis considerably to ref…
Deep learning methods have achieved great success in solving partial differential equations (PDEs), where the loss is often defined as an integral. The accuracy and efficiency of these algorithms depend greatly on the quadrature method. We propose to apply quasi-Monte Carlo (QMC) methods to the Deep Ritz Method (DRM) for solving the Neumann problems for the Poisson equation and the static Schrödinger equation. For error estimation, we decompose the error of using the deep learning algorithm to solve PDEs into the generalization error, the approximation error and the training error. We establis…
We propose a First-Order System Least Squares (FOSLS) method based on deep-learning for numerically solving second-order elliptic PDEs. The method we propose is capable of dealing with either variational and non-variational problems, and because of its meshless nature, it can also deal with problems posed in high-dimensional domains. We prove the $Γ$-convergence of the neural network approximation towards the solution of the continuous problem, and extend the convergence proof to some well-known related methods. Finally, we present several numerical examples illustrating the performance of our…
We present a new approach to compute selected eigenvalues and eigenvectors of the two-parameter eigenvalue problem. Our method requires computing generalized eigenvalue problems of the same size as the matrices of the initial two-parameter eigenvalue problem. The method is applicable for right definite problems, possibly after performing an affine transformation. This includes a class of Helmholtz equations when separation of variables is applied. We provide a convergence proof for extremal eigenvalues and empirical evidence along with a local convergence proof for other eigenvalues.
We present a stationary iteration method, namely Alternating Symmetric positive definite and Scaled symmetric positive semidefinite Splitting (ASSS), for solving the system of linear equations obtained by using finite element discretization of a distributed optimal control problem together with time-periodic parabolic equations. An upper bound for the spectral radius of the iteration method is given which is always less than 1. So convergence of the ASSS iteration method is guaranteed. The induced ASSS preconditioner is applied to accelerate the convergence speed of the GMRES method for solvin…
In the realm of big data and machine learning, data-parallel, distributed stochastic algorithms have drawn significant attention in the present days.~While the synchronous versions of these algorithms are well understood in terms of their convergence, the convergence analyses of their asynchronous counterparts are not widely studied. In this paper, we propose and analyze a {\it distributed, asynchronous parallel} SGD in light of solving an arbitrary consistent linear system by reformulating the system into a stochastic optimization problem as studied by Richtárik and Takác in [35]. We compare …
This paper presents and analyses a Constraint Energy Minimization Generalized Multiscale Finite Element Method (CEM-GMsFEM) for solving single-phase non-linear compressible flows in highly heterogeneous media. The construction of CEM-GMsFEM hinges on two crucial steps: First, the auxiliary space is constructed by solving local spectral problems, where the basis functions corresponding to small eigenvalues are captured. Then the basis functions are obtained by solving local energy minimization problems over the oversampling domains using the auxiliary space. The basis functions have exponential…
For a linear complementarity problem, we present a relaxaiton accelerated two-sweep matrix splitting iteration method. The convergence analysis illustrates that the proposed method converges to the exact solution of the linear complementarity problem when the system matrix is an $H_+$-matrix and the convergence conditions are given. Numerical experiments show that the proposed method is more efficient than the existing ones.
Deep learning and (deep) neural networks are emerging tools to address inverse problems and image reconstruction tasks. Despite outstanding performance, the mathematical analysis for solving inverse problems by neural networks is mostly missing. In this paper, we introduce and rigorously analyze families of deep regularizing neural networks (RegNets) of the form $B_α+ N_{θ(α)} B_α$, where $B_α$ is a classical regularization and the network $N_{θ(α)} B_α$ is trained to recover the missing part $\operatorname{Id}_X - B_α$ not found by the classical regularization. We show that these regularizing…
An optimal three-point eighth-order iterative method without memory for solving nonlinear equations with its dynamics
Technische Universität Dresden（德累斯顿工业大学）；Universiti Putra Malaysia（马来西亚博特拉大学）；MEDAlics, Research Center at Università per Stranieri Dante Alighieri（但丁外语大学MEDAlics研究中心）；Universidad de La Rioja（拉里奥哈大学）
