Acceleration techniques for regularized Newton methods applied to electromagnetic inverse medium scattering problems
Institut für Numerische und Angewandte Mathematik, Universität Göttingen（哥廷根大学数值与应用数学研究所）；Deutsches Zentrum für Luft- und Raumfahrt e. V. in der Helmholtz-Gemeinschaft, Institut für Aerodynamik und Strömungstechnik（德国航空航天中心（戴姆勒-克莱斯勒-阿利安特-马瑟里基金会），空气动力学与流动技术研究所）
We study the construction and updating of spectral preconditioners for regularized Newton methods and their application to electromagnetic inverse medium scattering problems. Moreover, we show how a Lepskiĭ-type stopping rule can be implemented efficiently for these methods. In numerical examples, the proposed method compares favorably with other iterative regularization method in terms of work-precision diagrams for exact data. For data perturbed by random noise, the Lepskiĭ-type stopping rule performs considerably better than the commonly used discrepancy principle.
In this paper we consider the iteratively regularized Gauss-Newton method, where regularization is achieved by Ivanov regularization, i.e., by imposing a priori constraints on the solution. We propose an a posteriori choice of the regularization radius, based on an inexact Newton / discrepancy principle approach, prove convergence and convergence rates under a variational source condition as the noise level tends to zero, and provide an analysis of the discretization error. Our results are valid in general, possibly nonreflexive Banach spaces, including, e.g., $L^\infty$ as a preimage space. T…
This paper studies a finite element discretization of the regularized Bingham equations that describe viscoplastic flow. An efficient nonlinear solver for the discrete model is then proposed and analyzed. The solver is based on Anderson acceleration (AA) applied to a Picard iteration, and we show accelerated convergence of the method by applying AA theory (recently developed by the authors) to the iteration, after showing sufficient smoothness properties of the associated fixed point operator. Numerical tests of spatial convergence are provided, as are results of the model for 2D and 3D driven…
This paper deals with regularized Newton methods, a flexible class of unconstrained optimization algorithms that is competitive with line search and trust region methods and potentially combines attractive elements of both. The particular focus is on combining regularization with limited memory quasi-Newton methods by exploiting the special structure of limited memory algorithms. Global convergence of regularization methods is shown under mild assumptions and the details of regularized limited memory quasi-Newton updates are discussed including their compact representations. Numerical results …
This paper studies the performance Newton's iteration applied with Anderson acceleration for solving the incompressible steady Navier-Stokes equations. We manifest that this method converges superlinearly with a good initial guess, and moreover, a large Anderson depth decelerates the convergence speed comparing to a small Anderson depth. We observe that the numerical tests confirm these analytical convergence results, and in addition, Anderson acceleration sometimes enlarges the domain of convergence for Newton's method.
For numerous parameter and state estimation problems, assimilating new data as they become available can help produce accurate and fast inference of unknown quantities. While most existing algorithms for solving those kind of ill-posed inverse problems can only be used with a single instance of the observed data, in this work we propose a new framework that enables existing algorithms to invert multiple instances of data in a sequential fashion. Specifically we will work with the well-known iteratively regularized Gauss-Newton method (IRGNM), a variational methodology for solving nonlinear inv…
Picard-Based Acceleration of Newton Continuation for Mean Field Game PDE Systems
New York University Shanghai（纽约大学上海）；NYU-ECNU Institute of Mathematical Sciences at NYU Shanghai（纽约大学上海华东师范大学数学科学联合研究院）；Department of Mathematics, New York University Shanghai（纽约大学上海数学系）；Department of Data Science, New York University Shanghai（纽约大学上海数据科学系）；Department of Mathematics, New York University（纽约大学数学系）
Optimization acceleration techniques such as momentum play a key role in state-of-the-art machine learning algorithms. Recently, generic vector sequence extrapolation techniques, such as regularized nonlinear acceleration (RNA) of Scieur et al., were proposed and shown to accelerate fixed point iterations. In contrast to RNA which computes extrapolation coefficients by (approximately) setting the gradient of the objective function to zero at the extrapolated point, we propose a more direct approach, which we call direct nonlinear acceleration (DNA). In DNA, we aim to minimize (an approximation…
