首页/论文/2002.09301

arXiv 2002.093012020-02stat.MLcs.LGcs.NAmath.NAstat.ME

Differentiable Likelihoods for Fast Inversion of 'Likelihood-Free' Dynamical Systems

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Bosch Center for Artificial Intelligence（博世人工智能中心）

Hans Kersting, Nicholas Krämer, Martin Schiegg, Christian Daniel, Michael Tiemann, Philipp Hennig

备注：: 11 pages (+ 5 pages appendix), 6 figures

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Likelihood-free (a.k.a. simulation-based) inference problems are inverse problems with expensive, or intractable, forward models. ODE inverse problems are commonly treated as likelihood-free, as their forward map has to be numerically approximated by an ODE solver. This, however, is not a fundamental constraint but just a lack of functionality in classic ODE solvers, which do not return a likelihood but a point estimate. To address this shortcoming, we employ Gaussian ODE filtering (a probabilistic numerical method for ODEs) to construct a local Gaussian approximation to the likelihood. This approximation yields tractable estimators for the gradient and Hessian of the (log-)likelihood. Insertion of these estimators into existing gradient-based optimization and sampling methods engenders new solvers for ODE inverse problems. We demonstrate that these methods outperform standard likelihood-free approaches on three benchmark-systems.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Sparse inversion for derivative of log determinant

Algorithms for Gaussian process, marginal likelihood methods or restricted maximum likelihood methods often require derivatives of log determinant terms. These log determinants are usually parametric with variance parameters of the underlying statistical models. This paper demonstrates that, when the underlying matrix is sparse, how to take the advantage of sparse inversion---selected inversion which share the same sparsity as the original matrix---to accelerate evaluating the derivative of log determinant.

2019-11-02cs.NAmath.NAstat.CO

马尔可夫等价性与可微结构学习中的一致性

Markov Equivalence and Consistency in Differentiable Structure Learning

Booth School of Business, The University of Chicago（芝加哥大学布斯商学院）；Machine Learning Department, Carnegie Mellon University（卡内基梅隆大学机器学习系）

针对现有DAG可微结构学习依赖强可识别性假设及优化器易利用损失伪影的问题，通过正则化似然识别马尔可夫等价类中最稀疏模型，推广至一般模型并经实验验证，为通用场景下的可微结构学习提供理论与方法支持。

2024-11-27cs.LGmath.STstat.ME

Variational Inference over Non-differentiable Cardiac Simulators using Bayesian Optimization

Performing inference over simulators is generally intractable as their runtime means we cannot compute a marginal likelihood. We develop a likelihood-free inference method to infer parameters for a cardiac simulator, which replicates electrical flow through the heart to the body surface. We improve the fit of a state-of-the-art simulator to an electrocardiogram (ECG) recorded from a real patient.

2017-12-09stat.ML

Green's function based unparameterised multi-dimensional kernel density and likelihood ratio estimator

This paper introduces a probability density estimator based on Green's function identities. A density model is constructed under the sole assumption that the probability density is differentiable. The method is implemented as a binary likelihood estimator for classification purposes, so issues such as mis-modeling and overtraining are also discussed. The identity behind the density estimator can be interpreted as a real-valued, non-scalar kernel method which is able to reconstruct differentiable density functions.

2012-04-18math.STstat.MLstat.TH

Diffsurv：面向删失生存时间数据的可微排序

Diffsurv: Differentiable sorting for censored time-to-event data

University College London（伦敦大学学院）；Berlin Institute of Health（柏林健康研究所）；BenevolentAI（BenevolentAI公司）

本文提出 Diffsurv，将可微排序扩展到含删失的生存时间数据，通过可能排列矩阵建模标签不确定性，并在风险预测及 top-k 风险预测中优于现有方法。

2023-04-26cs.LGstat.ML

Differentiable Annealed Importance Sampling and the Perils of Gradient Noise

University of Toronto（多伦多大学）；Vector Institute（矢量研究所）；Stanford University（斯坦福大学）

Annealed importance sampling (AIS) and related algorithms are highly effective tools for marginal likelihood estimation, but are not fully differentiable due to the use of Metropolis-Hastings correction steps. Differentiability is a desirable property as it would admit the possibility of optimizing marginal likelihood as an objective using gradient-based methods. To this end, we propose Differentiable AIS (DAIS), a variant of AIS which ensures differentiability by abandoning the Metropolis-Hastings corrections. As a further advantage, DAIS allows for mini-batch gradients. We provide a detailed…

2021-10-26cs.LGstat.ML

更多相关论文收起更多论文

Semi-Discrete Normalizing Flows through Differentiable Tessellation

Meta AI

Mapping between discrete and continuous distributions is a difficult task and many have had to resort to heuristical approaches. We propose a tessellation-based approach that directly learns quantization boundaries in a continuous space, complete with exact likelihood evaluations. This is done through constructing normalizing flows on convex polytopes parameterized using a simple homeomorphism with an efficient log determinant Jacobian. We explore this approach in two application settings, mapping from discrete to continuous and vice versa. Firstly, a Voronoi dequantization allows automaticall…

2022-12-11cs.LGstat.ML

重访可微结构学习：$\ell_1$ 惩罚的不一致性及其他

Revisiting Differentiable Structure Learning: Inconsistency of $\ell_1$ Penalty and Beyond

University of California San Diego（加利福尼亚大学圣迭戈分校）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）

本文揭示可微结构学习中 $\ell_1$ 惩罚似然在 Markov 等价类识别上的根本不一致，并提出结合 moral graph 约束与硬无环约束的 $\ell_0$ 混合方法。

2024-10-24cs.LGstat.ML

Fitting summary statistics of neural data with a differentiable spiking network simulator

École polytechnique fédérale de Lausanne (EPFL)（洛桑联邦理工学院）

Fitting network models to neural activity is an important tool in neuroscience. A popular approach is to model a brain area with a probabilistic recurrent spiking network whose parameters maximize the likelihood of the recorded activity. Although this is widely used, we show that the resulting model does not produce realistic neural activity. To correct for this, we suggest to augment the log-likelihood with terms that measure the dissimilarity between simulated and recorded activity. This dissimilarity is defined via summary statistics commonly used in neuroscience and the optimization is eff…

2021-11-14cs.LGq-bio.NCstat.ML

使用可微覆盖概率校准神经仿真推断

Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability

University of Geneva（日内瓦大学）；HES-SO/HEG Genève（瑞士西部高等专业学院/日内瓦管理学院）；The University of Tokyo（东京大学）；RIKEN（理化学研究所）；University of Liège（列日大学）

针对仿真推断中现有算法可能产生过度自信后验的问题，提出一种将可微覆盖概率校准项直接纳入神经模型训练目标的方法，实现端到端反向传播，在六个基准上取得更优的覆盖率和后验密度。

2023-10-20cs.LGstat.ML

Recurrent machines for likelihood-free inference

KTH Royal Institute of Technology（瑞典皇家理工学院）；University of Liège（列日大学）

Likelihood-free inference is concerned with the estimation of the parameters of a non-differentiable stochastic simulator that best reproduce real observations. In the absence of a likelihood function, most of the existing inference methods optimize the simulator parameters through a handcrafted iterative procedure that tries to make the simulated data more similar to the observations. In this work, we explore whether meta-learning can be used in the likelihood-free context, for learning automatically from data an iterative optimization procedure that would solve likelihood-free inference prob…

2019-01-02cs.LGhep-phphysics.data-an

Stochastic seismic waveform inversion using generative adversarial networks as a geological prior

Imperial College London（帝国理工学院）

We present an application of deep generative models in the context of partial-differential equation (PDE) constrained inverse problems. We combine a generative adversarial network (GAN) representing an a priori model that creates subsurface geological structures and their petrophysical properties, with the numerical solution of the PDE governing the propagation of acoustic waves within the earth's interior. We perform Bayesian inversion using an approximate Metropolis-adjusted Langevin algorithm (MALA) to sample from the posterior given seismic observations. Gradients with respect to the model…

2018-06-10cs.CVphysics.geo-phstat.ML

Darcy流反演中高效代理模型构建的序贯贝叶斯设计

Sequential Bayesian Design for Efficient Surrogate Construction in the Inversion of Darcy Flows

School of Mathematics and Statistics（数学与统计学学院）；Central South University（中南大学）；Institute of Mathematics（数学研究所）；Henan Academy of Sciences（河南省科学院）

针对Darcy流PDE反演中正演求解昂贵、全局代理构建成本高的问题，提出SBD-LAS，用序贯贝叶斯设计构建聚焦似然高概率区的局部精确代理并加速计算。

2025-07-23cs.LGstat.ML

Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach

We study statistical inference and distributionally robust solution methods for stochastic optimization problems, focusing on confidence intervals for optimal values and solutions that achieve exact coverage asymptotically. We develop a generalized empirical likelihood framework---based on distributional uncertainty sets constructed from nonparametric $f$-divergence balls---for Hadamard differentiable functionals, and in particular, stochastic optimization problems. As consequences of this theory, we provide a principled method for choosing the size of distributional uncertainty regions to pro…

2018-06-30stat.ML