In this paper we study a history matching approach that consists of finding stable approximations to the problem of minimizing the weighted least-squares functional that penalizes the misfit between the reservoir model predictions $G(u)$ and noisy observations $y^η$. In other words, we are interested in computing $u^η\equiv \arg\min_{u\in X}\frac{1}{2}\vert\vert Γ^{-1/2}(y-G(u))\vert\vert_{Y}^{2} $ where $Γ$ is the measurements error covariance, $Y$ is the observation space and $X$ is a set of admissible parameters. This is an ill-posed nonlinear inverse problem that we address by means of the regularizing Levenberg-Marquardt scheme developed in \cite{Hanke,Hanke2}. Under certain conditions on $G$, the theory of \cite{Hanke,Hanke2} ensures convergence of the scheme to stable approximations to the inverse problem. We propose an implementation of the regularizing Levenberg-Marquardt scheme that enforces prior knowledge of the geologic properties. In particular, the prior mean $\overline{u}$ is incorporated in the initial guess of the algorithm and the prior error covariance $C$ is enforced through the definition of the parameter space $X$. Our main goal is to numerically show that the proposed implementation of the regularizing Levenberg-Marquardt scheme of Hanke is a robust method capable of providing accurate estimates of the geologic properties for small noise measurements. The performance for recovering the true permeability with the regularizing Levenberg-Marquardt scheme is compared against the more standard techniques for history matching proposed in \cite{Li,Tavakoli,svdRML,Oliver}. Our numerical experiments suggest that the history matching approach based on iterative regularization is robust and could potentially be used to improve further on various methodologies already proposed as effective tools for history matching in petroleum reservoirs
The Levenberg-Marquardt algorithm is one of the most popular algorithms for finding the solution of nonlinear least squares problems. Across different modified variations of the basic procedure, the algorithm enjoys global convergence, a competitive worst case iteration complexity rate, and a guaranteed rate of local convergence for both zero and nonzero small residual problems, under suitable assumptions. We introduce a novel Levenberg-Marquardt method that matches, simultaneously, the state of the art in all of these convergence properties with a single seamless algorithm. Numerical experime…
In this paper, we first propose a new Levenberg-Marquardt method for solving constrained (and not necessarily square) nonlinear systems. Basically, the method combines the unconstrained Levenberg-Marquardt method with a type of feasible inexact projection. The local convergence of the new method as well as results on its rate are established by using an error bound condition, which is weaker than the standard full-rank assumption. We further present and analyze a global version of the first method by means of a nonmonotone line search technique. Finally, numerical experiments illustrating the …
We herein propose a variant of the projected inexact Levenberg--Marquardt method (ILMM) for solving constrained nonsmooth equations. Since the orthogonal projection onto the feasible set may be computationally expensive, we propose a local ILMM with feasible inexact projections. By using assumption of semi-smoothness and an error bound condition which is weaker than the standard full rank assumption, we establish its convergence results, including results on its rate. Finally, some computacional results are reported in order to illustrate the advantages of the new schemes.
We consider the optimistic bilevel optimization problem, known to have a wide range of applications in engineering, that we transform into a single-level optimization problem by means of the lower-level optimal value function reformulation. Subsequently, based on the partial calmness concept, we build an equation system, which is parameterized by the corresponding partial exact penalization parameter. We then design and analyze a Levenberg-Marquardt method to solve this parametric system of equations. Considering the fact that the selection of the partial exact penalization parameter is a crit…
Globally convergent variants of the Gauss-Newton algorithm are often the methods of choice to tackle nonlinear least-squares problems. Among such frameworks, Levenberg-Marquardt and trust-region methods are two well-established, similar paradigms. Both schemes have been studied when the Gauss-Newton model is replaced by a random model that is only accurate with a given probability. Trust-region schemes have also been applied to problems where the objective value is subject to noise: this setting is of particular interest in fields such as data assimilation, where efficient methods that can ada…
We describe and analyse Levenberg-Marquardt methods for solving systems of nonlinear equations. More specifically, we propose an adaptive formula for the Levenberg-Marquardt parameter and analyse the local convergence of the method under Hölder metric subregularity of the function defining the equation and Hölder continuity of its gradient mapping. Further, we analyse the local convergence of the method under the additional assumption that the Łojasiewicz gradient inequality holds. We finally report encouraging numerical results confirming the theoretical findings for the problem of computing …
A q-Levenberg-Marquardt method is an iterative procedure that blends a q-steepest descent and q-Gauss-Newton methods. When the current solution is far from the correct one the algorithm acts as the q-steepest descent method. Otherwise the algorithm acts as the q-Gauss-Newton method. A damping parameter is used to interpolate between these two methods. The q-parameter is used to escape from local minima and to speed up the search process near the optimal solution.
In this paper, we consider a modified Levenberg--Marquardt method for solving an ill-posed inverse problem where the forward mapping is not Gâteaux differentiable. By relaxing the standard assumptions for the classical smooth setting, we derive asymptotic stability estimates that are then used to prove the convergence of the proposed method. This method can be applied to an inverse source problem for a non-smooth semilinear elliptic PDE where a Bouligand subdifferential can be used in place of the non-existing Fréchet derivative, and we show that the corresponding Bouligand-Levenberg-Marquardt…
We present two globally convergent Levenberg-Marquardt methods for finding zeros of Hölder metrically subregular mappings that may have non-isolated zeros. The first method unifies the Levenberg- Marquardt direction and an Armijo-type line search, while the second incorporates this direction with a nonmonotone trust-region technique. For both methods, we prove the global convergence to a first-order stationary point of the associated merit function. Furthermore, the worst-case global complexity of these methods are provided, indicating that an approximate stationary point can be computed in at…
We consider large-scale nonlinear least squares problems with sparse residuals, each of them depending on a small number of variables. A decoupling procedure which results in a splitting of the original problems into a sequence of independent problems of smaller sizes is proposed and analysed. The smaller size problems are modified in a way that offsets the error made by disregarding dependencies that allow us to split the original problem. The resulting method is a modification of the Levenberg-Marquardt method with smaller computational costs. Global convergence is proved as well as local li…
In connection with the optimization problem $$\inf_{x\in argmin Ψ}\{Φ(x)+Θ(x)\},$$ where $Φ$ is a proper, convex and lower semicontinuous function and $Θ$ and $Ψ$ are convex and smooth functions defined on a real Hilbert space, we investigate the asymptotic behavior of the trajectories of the nonautonomous Levenberg-Marquardt dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partialΦ(x(t))\\ λ(t)\dot x(t) + \dot v(t) + v(t) + \nabla Θ(x(t))+β(t)\nabla Ψ(x(t))=0, \end{array}\right.\end{equation*} where $λ$ and $β$ are functions of time controlling the velocity and the penalty …
We develop a Levenberg-Marquardt method for minimizing the sum of a smooth nonlinear least-squar es term $f(x) = \tfrac{1}{2} \|F(x)\|_2^2$ and a nonsmooth term $h$. Both $f$ and $h$ may be nonconvex. Steps are computed by minimizing the sum of a regularized linear least-squares model and a model of $h$ using a first-order method such as the proximal gradient method. We establish global convergence to a first-order stationary point of both a trust-region and a regularization variant of the Levenberg-Marquardt method under the assumptions that $F$ and its Jacobian are Lipschitz continuous and $…
The Levenberg-Marquardt (LM) method is commonly used for inverting models used to describe geothermal, groundwater, or oil and gas reservoirs. In previous studies LM parameter updates have been made tractable for highly parameterized inverse problems with large data sets by applying matrix factorization methods or iterative linear solvers to approximately solve the update equations. Some studies have shown that basing model updates on the truncated singular value decomposition (TSVD) of a dimensionless sensitivity matrix achieved using Lanczos iteration can speed up the inversion of reservoir …
Defence Science and Technology Group（国防科技集团）；CSIRO, Data61（澳大利亚联邦科学与工业研究组织 Data61）；Australian National University（澳大利亚国立大学）
The Levenberg-Marquardt (LM) optimization algorithm has been widely used for solving machine learning problems. Literature reviews have shown that the LM can be very powerful and effective on moderate function approximation problems when the number of weights in the network is not more than a couple of hundred. In contrast, the LM does not seem to perform as well when dealing with pattern recognition or classification problems, and inefficient when networks become large (e.g. with more than 500 weights). In this paper, we exploit the true power of LM algorithm using some real world aircraft da…
On the approximation of the solution of partial differential equations by artificial neural networks trained by a multilevel Levenberg-Marquardt method
INPT-IRIT, University of Toulouse and ENSEEIHT（INPT-IRIT，图卢兹大学和ENSEEIHT）；ISAE-SUPAERO, University of Toulouse（ISAE-SUPAERO，图卢兹大学）
Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
organization= Department of Mathematics, Institut Teknologi Sepuluh Nopember , city= Surabaya , postcode= 60111 , country= Indonesia；organization= Department of Mathematics, Khalifa University of Science \& Technology , city= Abu Dhabi , postcode= PO Box 127788 , country= United Arab Emirates；organization= Department of Mathematics, Faculty of Mathematics
We propose two variants of Newton method for solving unconstrained minimization problem. Our method leverages optimization techniques such as penalty and augmented Lagrangian method to generate novel variants of the Newton method namely the Penalty Newton method and the Augmented Newton method. In doing so, we recover several well-known existing Newton method variants such as Damped Newton, Levenberg, and Levenberg-Marquardt methods as special cases. Moreover, the proposed Augmented Newton method can be interpreted as Newton method with adaptive heavy ball momentum. We provide global convergen…
The focus of this work is on an alternative implementation of the iterative ensemble smoother (iES). We show that iteration formulae similar to those used in \cite{chen2013-levenberg,emerick2012ensemble} can be derived by adopting a regularized Levenberg-Marquardt (RLM) algorithm \cite{jin2010regularized} to approximately solve a minimum-average-cost (MAC) problem. This not only leads to an alternative theoretical tool in understanding and analyzing the behaviour of the aforementioned iES, but also provides insights and guidelines for further developments of the smoothing algorithms. For illus…
