ABC Samplers

可逆跳转马尔可夫链蒙特卡罗与多模型采样器

Reversible jump Markov chain Monte Carlo and multi-model samplers

本文将作为章节发表于MCMC手册第二版，探讨可逆跳转马尔可夫链蒙特卡罗与多模型采样器。

Likelihood-free approximate Gibbs sampling

Likelihood-free methods such as approximate Bayesian computation (ABC) have extended the reach of statistical inference to problems with computationally intractable likelihoods. Such approaches perform well for small-to-moderate dimensional problems, but suffer a curse of dimensionality in the number of model parameters. We introduce a likelihood-free approximate Gibbs sampler that naturally circumvents the dimensionality issue by focusing on lower-dimensional conditional distributions. These distributions are estimated by flexible regression models either before the sampler is run, or adaptiv…

分层距离空间提升近似贝叶斯计算中顺序采样器的效率

Stratified distance space improves the efficiency of sequential samplers for approximate Bayesian computation

本文提出分层距离ABC SMC方法，通过按粒子距离分层构建提议分布并引入新停止规则，显著提升顺序采样器在近似贝叶斯计算中的接受率与效率。

Summary Statistics in Approximate Bayesian Computation

This document is due to appear as a chapter of the forthcoming Handbook of Approximate Bayesian Computation (ABC) edited by S. Sisson, Y. Fan, and M. Beaumont. Since the earliest work on ABC, it has been recognised that using summary statistics is essential to produce useful inference results. This is because ABC suffers from a curse of dimensionality effect, whereby using high dimensional inputs causes large approximation errors in the output. It is therefore crucial to find low dimensional summaries which are informative about the parameter inference or model choice task at hand. This chapte…

粒子吉布斯用于状态空间模型的似然无关推断及其在随机波动率中的应用

Particle Gibbs for Likelihood-Free Inference of State Space Models with Application to Stochastic Volatility

针对似然难处理的状态空间模型，提出结合条件辅助粒子滤波的粒子吉布斯采样器，在ABC框架下实现参数推断，并在随机波动率模型及S&P 500数据上验证了准确性。

Approximating the Likelihood in Approximate Bayesian Computation

This chapter will appear in the forthcoming Handbook of Approximate Bayesian Computation (2018). The conceptual and methodological framework that underpins approximate Bayesian computation (ABC) is targetted primarily towards problems in which the likelihood is either challenging or missing. ABC uses a simulation-based non-parametric estimate of the likelihood of a summary statistic and assumes that the generation of data from the model is computationally cheap. This chapter reviews two alternative approaches for estimating the intractable likelihood, with the goal of reducing the necessary mo…

使用早期拒绝马尔可夫链蒙特卡罗和高斯过程加速ABC方法

Using early rejection Markov chain Monte Carlo and Gaussian processes to accelerate ABC methods

针对似然函数不可用的近似贝叶斯计算（ABC）在复杂模型中计算成本高的问题，提出基于高斯过程的早期拒绝马尔可夫链蒙特卡罗（ejMCMC）采样器，通过差异模型提前拒绝样本以减少不必要的合成数据生成，显著提升了推断效率。

Delayed acceptance ABC-SMC

Approximate Bayesian computation (ABC) is now an established technique for statistical inference used in cases where the likelihood function is computationally expensive or not available. It relies on the use of a~model that is specified in the form of a~simulator, and approximates the likelihood at a~parameter value $θ$ by simulating auxiliary data sets $x$ and evaluating the distance of $x$ from the true data $y$. However, ABC is not computationally feasible in cases where using the simulator for each $θ$ is very expensive. This paper investigates this situation in cases where a~cheap, but a…

Adaptive Approximate Bayesian Computation Tolerance Selection

Approximate Bayesian Computation (ABC) methods are increasingly used for inference in situations in which the likelihood function is either computationally costly or intractable to evaluate. Extensions of the basic ABC rejection algorithm have improved the computational efficiency of the procedure and broadened its applicability. The ABC-Population Monte Carlo (ABC-PMC) approach of Beaumont et al. (2009) has become a popular choice for approximate sampling from the posterior. ABC-PMC is a sequential sampler with an iteratively decreasing value of the tolerance, which specifies how close the si…

Efficient learning in ABC algorithms

Approximate Bayesian Computation has been successfully used in population genetics to bypass the calculation of the likelihood. These methods provide accurate estimates of the posterior distribution by comparing the observed dataset to a sample of datasets simulated from the model. Although parallelization is easily achieved, computation times for ensuring a suitable approximation quality of the posterior distribution are still high. To alleviate the computational burden, we propose an adaptive, sequential algorithm that runs faster than other ABC algorithms but maintains accuracy of the appro…

面向随机微分方程ABC推断的数据条件模拟

Towards Data-Conditional Simulation for ABC Inference in Stochastic Differential Equations

我们提出一种基于条件模拟和不变神经网络学习汇总统计的ABC-SMC方法，用于随机微分方程推断，实现准确且快约三倍的推断。

astroABC: An Approximate Bayesian Computation Sequential Monte Carlo sampler for cosmological parameter estimation

Given the complexity of modern cosmological parameter inference where we are faced with non-Gaussian data and noise, correlated systematics and multi-probe correlated data sets, the Approximate Bayesian Computation (ABC) method is a promising alternative to traditional Markov Chain Monte Carlo approaches in the case where the Likelihood is intractable or unknown. The ABC method is called "Likelihood free" as it avoids explicit evaluation of the Likelihood by using a forward model simulation of the data which can include systematics. We introduce astroABC, an open source ABC Sequential Monte Ca…

Probably approximate Bayesian computation: nonasymptotic convergence of ABC under misspecification

Approximate Bayesian computation (ABC) is a widely used inference method in Bayesian statistics to bypass the point-wise computation of the likelihood. In this paper we develop theoretical bounds for the distance between the statistics used in ABC. We show that some versions of ABC are inherently robust to misspecification. The bounds are given in the form of oracle inequalities for a finite sample size. The dependence on the dimension of the parameter space and the number of statistics is made explicit. The results are shown to be amenable to oracle inequalities in parameter space. We apply o…

Optimal proposals for Approximate Bayesian Computation

We derive the optimal proposal density for Approximate Bayesian Computation (ABC) using Sequential Monte Carlo (SMC) (or Population Monte Carlo, PMC). The criterion for optimality is that the SMC/PMC-ABC sampler maximise the effective number of samples per parameter proposal. The optimal proposal density represents the optimal trade-off between favoring high acceptance rate and reducing the variance of the importance weights of accepted samples. We discuss two convenient approximations of this proposal and show that the optimal proposal density gives a significant boost in the expected samplin…

Coupling stochastic EM and Approximate Bayesian Computation for parameter inference in state-space models

We study the class of state-space models and perform maximum likelihood estimation for the model parameters. We consider a stochastic approximation expectation-maximization (SAEM) algorithm to maximize the likelihood function with the novelty of using approximate Bayesian computation (ABC) within SAEM. The task is to provide each iteration of SAEM with a filtered state of the system, and this is achieved using an ABC sampler for the hidden state, based on sequential Monte Carlo (SMC) methodology. It is shown that the resulting SAEM-ABC algorithm can be calibrated to return accurate inference, …

Likelihood-free Model Choice

This document is an invited chapter covering the specificities of ABC model choice, intended for the incoming Handbook of ABC by Sisson, Fan, and Beaumont (2017). Beyond exposing the potential pitfalls of ABC based posterior probabilities, the review emphasizes mostly the solution proposed by Pudlo et al. (2016) on the use of random forests for aggregating summary statistics and and for estimating the posterior probability of the most likely model via a secondary random fores.

Approximate methods for dynamic ecological models

This document is due to appear as a chapter of the forthcoming Handbook of Approximate Bayesian Computation (ABC) by S. Sisson, L. Fan, and M. Beaumont. Here we describe some of the circumstances under which statistical ecologists might benefit from using methods that base statistical inference on a set of summary statistics, rather than on the full data. We focus particularly on one such approach, Synthetic Likelihood, and we show how this method represents an alternative to particle filters, for the purpose of fitting State Space Models of ecological interest. As an example application, we c…

Accelerating inference for diffusions observed with measurement error and large sample sizes using Approximate Bayesian Computation

In recent years dynamical modelling has been provided with a range of breakthrough methods to perform exact Bayesian inference. However it is often computationally unfeasible to apply exact statistical methodologies in the context of large datasets and complex models. This paper considers a nonlinear stochastic differential equation model observed with correlated measurement errors and an application to protein folding modelling. An Approximate Bayesian Computation (ABC) MCMC algorithm is suggested to allow inference for model parameters within reasonable time constraints. The ABC algorithm us…

Markov chain Monte Carlo importance samplers for Bayesian models with intractable likelihoods

We consider the efficient use of an approximation within Markov chain Monte Carlo (MCMC), with subsequent importance sampling (IS) correction of the Markov chain inexact output, leading to asymptotically exact inference. We detail convergence and central limit theorems for the resulting MCMC-IS estimators. We also consider the case where the approximate Markov chain is pseudo-marginal, requiring unbiased estimators for its approximate marginal target. Convergence results with asymptotic variance formulae are shown for this case, and for the case where the IS weights based on unbiased estimator…

Optimizing Threshold - Schedules for Approximate Bayesian Computation Sequential Monte Carlo Samplers: Applications to Molecular Systems

The likelihood-free sequential Approximate Bayesian Computation (ABC) algorithms, are increasingly popular inference tools for complex biological models. Such algorithms proceed by constructing a succession of probability distributions over the parameter space conditional upon the simulated data lying in an $ε$--ball around the observed data, for decreasing values of the threshold $ε$. While in theory, the distributions (starting from a suitably defined prior) will converge towards the unknown posterior as $ε$ tends to zero, the exact sequence of thresholds can impact upon the computational ef…

Finding our Way in the Dark: Approximate MCMC for Approximate Bayesian Methods

With larger data at their disposal, scientists are emboldened to tackle complex questions that require sophisticated statistical models. It is not unusual for the latter to have likelihood functions that elude analytical formulations. Even under such adversity, when one can simulate from the sampling distribution, Bayesian analysis can be conducted using approximate methods such as Approximate Bayesian Computation (ABC) or Bayesian Synthetic Likelihood (BSL). A significant drawback of these methods is that the number of required simulations can be prohibitively large, thus severely limiting th…

一种基于全局-局部提议的自适应近似贝叶斯计算MCMC算法

An adaptive approximate Bayesian computation MCMC with Global-Local proposals

本文提出了一种结合全局和局部提议的自适应ABC-MCMC算法，通过改进的自适应方案提升采样效率和收敛可靠性。

基于GANs的生成贝叶斯推断

Generative Bayesian Inference with GANs

本文提出了一种基于GANs的贝叶斯推断方法，通过对抗性优化直接目标后验，并结合数据驱动提议和变分贝叶斯进行后处理，实验证明其在模拟数据中表现优异。

Marginal sequential Monte Carlo for doubly intractable models

Bayesian inference for models that have an intractable partition function is known as a doubly intractable problem, where standard Monte Carlo methods are not applicable. The past decade has seen the development of auxiliary variable Monte Carlo techniques (Møller et al., 2006; Murray et al., 2006) for tackling this problem; these approaches being members of the more general class of pseudo-marginal, or exact-approximate, Monte Carlo algorithms (Andrieu and Roberts, 2009), which make use of unbiased estimates of intractable posteriors. Everitt et al. (2017) investigated the use of exact-approx…

Transdimensional Approximate Bayesian Computation for Inference on Invasive Species Models with Latent Variables of Unknown Dimension

Accurate information on patterns of introduction and spread of non-native species is essential for making predictions and management decisions. In many cases, estimating unknown rates of introduction and spread from observed data requires evaluating intractable variable-dimensional integrals. In general, inference on the large class of models containing latent variables of large or variable dimension precludes exact sampling techniques. Approximate Bayesian computation (ABC) methods provide an alternative to exact sampling but rely on inefficient conditional simulation of the latent variables.…

Approximate maximum likelihood estimation using data-cloning ABC

A maximum likelihood methodology for a general class of models is presented, using an approximate Bayesian computation (ABC) approach. The typical target of ABC methods are models with intractable likelihoods, and we combine an ABC-MCMC sampler with so-called "data cloning" for maximum likelihood estimation. Accuracy of ABC methods relies on the use of a small threshold value for comparing simulations from the model and observed data. The proposed methodology shows how to use large threshold values, while the number of data-clones is increased to ease convergence towards an approximate maximum…

Computational Solutions for Bayesian Inference in Mixture Models

This chapter surveys the most standard Monte Carlo methods available for simulating from a posterior distribution associated with a mixture and conducts some experiments about the robustness of the Gibbs sampler in high dimensional Gaussian settings. This is a chapter prepared for the forthcoming 'Handbook of Mixture Analysis'.

Learning of state-space models with highly informative observations: a tempered Sequential Monte Carlo solution

Probabilistic (or Bayesian) modeling and learning offers interesting possibilities for systematic representation of uncertainty using probability theory. However, probabilistic learning often leads to computationally challenging problems. Some problems of this type that were previously intractable can now be solved on standard personal computers thanks to recent advances in Monte Carlo methods. In particular, for learning of unknown parameters in nonlinear state-space models, methods based on the particle filter (a Monte Carlo method) have proven very useful. A notoriously challenging problem,…

Technical Note: Approximate Bayesian parameterization of a process-based tropical forest model

Inverse parameter estimation of process-based models is a long-standing problem in many scientific disciplines. A key question for inverse parameter estimation is how to define the metric that quantifies how well model predictions fit to the data. This metric can be expressed by general cost or objective functions, but statistical inversion methods require a particular metric, the probability of observing the data given the model parameters, known as the likelihood. For technical and computational reasons, likelihoods for process-based stochastic models are usually based on general assumptions…

基于Gibbs步骤的分量近似贝叶斯计算

Component-wise approximate Bayesian computation via Gibbs-like steps

本文提出一种基于Gibbs步骤的分量近似贝叶斯计算方法，通过降维汇总统计量提升效率，实现更高效的参数估计。