An effective numerical method is presented for optimizing model parameters that can be applied to any type of system of non-linear equations and any number of data-points, which does not require explicit formulation of the objective function or its partial derivatives. The numerics are reduced to solving a non-linear least squares problem, which uses the Levenberg-Marquardt algorithm and the Jacobian is approximated by applying rank-one updates using Broyden's method. An advantage of this methodology over conventional approaches is that the partial derivatives of the objective function do not have to be analytically calculated. For instance, there may be situations where one cannot formulate the partial derivatives, such as cases involving an objective function that itself contains a nested optimization problem. Moreover, a line search algorithm is also described that ensures that the Armijo conditions are satisfied and that convergence is assured, which makes the success of the approach insensitive to the initial estimates of the model parameters. The foregoing numerical methods are described with respect to the development of the Optima software to solve inverse problems, which are reduced to non-linear least squares problems. This computational approach has proven to be particularly useful at solving inverse problems of very complex physical models that cannot be optimized directly in a practical way.
An efficient Quasi-Newton method for nonlinear inverse problems via learned singular values
University of Sheffield(谢菲尔德大学);Purdue University(普渡大学);University of Science and Technology of China(中国科学技术大学);University of Oulu(奥卢大学);University College London(伦敦大学学院)
hIPPYlib: An Extensible Software Framework for Large-Scale Inverse Problems Governed by PDEs; Part I: Deterministic Inversion and Linearized Bayesian Inference
Complete Deterministic Dynamics and Spectral Decomposition of the Linear Ensemble Kalman Inversion
解$y = A u +\varepsilon$型贝叶斯反问题的集合卡尔曼反演(EKI)是一种强有力的工具,它以$u$为未知参数,以$y$为给定数据,以$\varepsilon$为观测噪声,通常从序贯蒙特卡罗的观点导出.它描述了一个粒子的合奏$\{u^j(t)\}_{j=1}^J$,其初始的经验措施是从以前的采样,在一个人工时间$t$对逆问题的近似解的演变,与$t=1$仿真后,和$t\to\infty$对应的下正则化的最小范数解的逆问题的动态。使用谱技术,我们提供了一个完整的描述的确定性动力学的EKI及其在参数空间中的渐近行为。特别是,我们分析了天真的EKI和平均场EKI的动力学,特别关注他们的时间渐近行为。此外,我们表明-即使在确定性的情况下-参数空间中的残差不单调递减的欧几里德规范,并建议一个问题适应规范,其中单调性可以证明。最后,我们推导出一个系统的常微分方程的频谱和特征向量的协方差矩阵。虽然分析的目的是EKI,我们相信,它可以应用于了解更一般的粒子为基础的动力系统。