Inexact Newton regularization methods have been proposed by Hanke and Rieder for solving nonlinear ill-posed inverse problems. Every such a method consists of two components: an outer Newton iteration and an inner scheme providing increments by regularizing local linearized equations. The method is terminated by a discrepancy principle. In this paper we consider the inexact Newton regularization methods with the inner scheme defined by Landweber iteration, the implicit iteration, the asymptotic regularization and Tikhonov regularization. Under certain conditions we obtain the order optimal convergence rate result which improves the suboptimal one of Rieder. We in fact obtain a more general order optimality result by considering these inexact Newton methods in Hilbert scales.
Inexact Newton Methods are widely used to solve systems of nonlinear equations. The convergence of these methods is controlled by the relative linear tolerance, $η_ν$, that is also called the forcing term. A very small $η_ν$ may lead to oversolving the Newton equation. Practical reservoir simulation uses inexact Newton methods with fixed forcing term, usually in the order of $10^{-3}$ or $10^{-4}$. Alternatively, variable forcing terms for a given inexact Newton step have proved to be quite successful in reducing the degree of oversolving in various practical applications. The cumulative numbe…
This paper is concerned with the inverse problem of constructing a symmetric nonnegative matrix from realizable spectrum. We reformulate the inverse problem as an underdetermined nonlinear matrix equation over a Riemannian product manifold. To solve it, we develop a Riemannian underdetermined inexact Newton dogleg method for solving a general underdetermined nonlinear equation defined between Riemannian manifolds and Euclidean spaces. The global and quadratic convergence of the proposed method is established under some mild assumptions. Then we solve the inverse problem by applying the propose…
We study stochastic inexact Newton methods and consider their application in nonconvex settings. Building on the work of [R. Bollapragada, R. H. Byrd, and J. Nocedal, IMA Journal of Numerical Analysis, 39 (2018), pp. 545--578] we derive bounds for convergence rates in expected value for stochastic low rank Newton methods, and stochastic inexact Newton Krylov methods. These bounds quantify the errors incurred in subsampling the Hessian and gradient, as well as in approximating the Newton linear solve, and in choosing regularization and step length parameters. We deploy these methods in training…
In this work, we address the implementation and performance of inexact Newton-Krylov and quasi-Newton algorithms, more specifically the BFGS method, for the solution of the nonlinear elasticity equations, and compare them to a standard Newton-Krylov method. This is done through a systematic analysis of the performance of the solvers with respect to the problem size, the magnitude of the data and the number of processors in both almost incompressible and incompressible mechanics. We consider three test cases: Cook's membrane (static, almost incompressible), a twist test (static, incompressible)…
This paper extends the algorithm of Benner, Heinkenschloss, Saak, and Weichelt: An inexact low-rank Newton-ADI method for large-scale algebraic Riccati equations, Applied Numerical Mathematics Vol.~108 (2016), pp.~125--142, doi:10.1016/j.apnum.2016.05.006 to Riccati equations associated with Hessenberg index-2 Differential Algebratic Equation (DAE) systems. Such DAE systems arise, e.g., from semi-discretized, linearized (around steady state) Navier-Stokes equations. The solution of the associated Riccati equation is important, e.g., to compute feedback laws that stabilize the Navier-Stokes equ…
We analyse errors of randomized explicit and implicit Euler schemes for approximate solving of ordinary differential equations (ODEs). We consider classes of ODEs for which the right-hand side functions satisfy Lipschitz condition globally or only locally. Moreover, we assume that only inexact discrete information, corrupted by some noise, about the right-hand side function is available. Optimality and stability of explicit and implicit randomized Euler algorithms are also investigated.
This work blends the inexact Newton method with iterative combined approximations (ICA) for solving topology optimization problems under the assumption of geometric nonlinearity. The density-based problem formulation is solved using a sequential piecewise linear programming (SPLP) algorithm. Five distinct strategies have been proposed to control the frequency of the factorizations of the Jacobian matrices of the nonlinear equilibrium equations. Aiming at speeding up the overall iterative scheme while keeping the accuracy of the approximate solutions, three of the strategies also use an ICA sch…
We propose numerical algorithms for solving large deformation diffeomorphic image registration problems. We formulate the nonrigid image registration problem as a problem of optimal control. This leads to an infinite-dimensional partial differential equation (PDE) constrained optimization problem. The PDE constraint consists, in its simplest form, of a hyperbolic transport equation for the evolution of the image intensity. The control variable is the velocity field. Tikhonov regularization on the control ensures well-posedness. We consider standard smoothness regularization based on $H^1$- or …
We propose a stochastic first-order trust-region method with inexact function and gradient evaluations for solving finite-sum minimization problems. Using a suitable reformulation of the given problem, our method combines the inexact restoration approach for constrained optimization with the trust-region procedure and random models. Differently from other recent stochastic trust-region schemes, our proposed algorithm improves feasibility and optimality in a modular way. We provide the expected number of iterations for reaching a near-stationary point by imposing some probability accuracy requi…
This paper focuses on regularisation methods using models up to the third order to search for up to second-order critical points of a finite-sum minimisation problem. The variant presented belongs to the framework of [3]: it employs random models with accuracy guaranteed with a sufficiently large prefixed probability and deterministic inexact function evaluations within a prescribed level of accuracy. Without assuming unbiased estimators, the expected number of iterations is $\mathcal{O}\bigl(ε_1^{-2}\bigr)$ or $\mathcal{O}\bigl(ε_1^{-{3/2}}\bigr)$ when searching for a first-order critical poi…
In this paper we consider the iteratively regularized Gauss-Newton method, where regularization is achieved by Ivanov regularization, i.e., by imposing a priori constraints on the solution. We propose an a posteriori choice of the regularization radius, based on an inexact Newton / discrepancy principle approach, prove convergence and convergence rates under a variational source condition as the noise level tends to zero, and provide an analysis of the discretization error. Our results are valid in general, possibly nonreflexive Banach spaces, including, e.g., $L^\infty$ as a preimage space. T…
As second-order methods, Gauss--Newton-type methods can be more effective than first-order methods for the solution of nonsmooth optimization problems with expensive-to-evaluate smooth components. Such methods, however, often do not converge. Motivated by nonlinear inverse problems with nonsmooth regularization, we propose a new Gauss--Newton-type method with inexact relaxed steps. We prove that the method converges to a set of disjoint critical points given that the linearisation of the forward operator for the inverse problem is sufficiently precise. We extensively evaluate the performance o…
We consider an optimal control problem for the steady-state Kirchhoff equation, a prototype for nonlocal partial differential equations, different from fractional powers of closed operators. Existence and uniqueness of solutions of the state equation, existence of global optimal solutions, differentiability of the control-to-state map and first-order necessary optimality conditions are established. The aforementioned results require the controls to be functions in $H^1$ and subject to pointwise upper and lower bounds. In order to obtain the Newton differentiability of the optimality conditions…
The sequential fully implicit (SFI) scheme was introduced (Jenny et al. 2006) for solving coupled flow and transport problems. Each time step for SFI consists of an outer loop, in which there are inner Newton loops to implicitly and sequentially solve the pressure and transport sub-problems. In standard SFI, the sub-problems are usually solved with tight tolerances at every outer iteration. This can result in wasted computations that contribute little progress towards the coupled solution. The issue is known as `over-solving'. Our objective is to minimize the cost of inner solvers while mainta…
