We consider time series data modeled by ordinary differential equations (ODEs), widespread models in physics, chemistry, biology and science in general. The sensitivity analysis of such dynamical systems usually requires calculation of various derivatives with respect to the model parameters.
We employ the adjoint state method (ASM) for efficient computation of the first and the second derivatives of likelihood functionals constrained by ODEs with respect to the parameters of the underlying ODE model. Essentially, the gradient can be computed with a cost (measured by model evaluations) that is independent of the number of the ODE model parameters and the Hessian with a linear cost in the number of the parameters instead of the quadratic one. The sensitivity analysis becomes feasible even if the parametric space is high-dimensional.
The main contributions are derivation and rigorous analysis of the ASM in the statistical context, when the discrete data are coupled with the continuous ODE model. Further, we present a highly optimized implementation of the results and its benchmarks on a number of problems.
The results are directly applicable in (e.g.) maximum-likelihood estimation or Bayesian sampling of ODE based statistical models, allowing for faster, more stable estimation of parameters of the underlying ODE model.
StAD: Stein Amortized Divergence for Fast Likelihoods with Diffusion and Flow
Institute of Astronomy(天文研究所);Kavli Institute for Cosmology, University of Cambridge, Cambridge, UK(剑桥大学卡文迪许实验室,剑桥,英国);Research Computing Services, University of Cambridge, Cambridge, UK(剑桥大学研究计算服务);The Oskar Klein Centre, Department of Physics, Stockholm University, Stockholm, SE(斯德哥尔摩大学物理系奥斯卡·克莱因中心);Cavendish Laboratory, Department of Physics, University of Cambridge, Cambridge, UK(剑桥大学物理系卡文迪许实验室)
Change Detection in Probability Flow ODE: Online Testing in Diffusion Latent Spaces
National Research University Higher School of Economics(国立研究型大学高等经济学院);The University of Sydney Business School(悉尼大学商学院);CIREQ(蒙特利尔数量经济学研究中心);CEBDA(应用经济学与数据分析研究中心)
Flow-based generative models as iterative algorithms in probability space
H. Milton Stewart School of Industrial and Systems Engineering (ISyE) at the Georgia Institute of Technology(佐治亚理工学院H.米尔顿·斯图尔特工业与系统工程学院);Mathematics Department at Duke University(杜克大学数学系)