Bayesian inference involves two main computational challenges. First, in estimating the parameters of some model for the data, the posterior distribution may well be highly multi-modal: a regime in which the convergence to stationarity of traditional Markov Chain Monte Carlo (MCMC) techniques becomes incredibly slow. Second, in selecting between a set of competing models the necessary estimation of the Bayesian evidence for each is, by definition, a (possibly high-dimensional) integration over the entire parameter space; again this can be a daunting computational task, although new Monte Carlo (MC) integration algorithms offer solutions of ever increasing efficiency. Nested sampling (NS) is one such contemporary MC strategy targeted at calculation of the Bayesian evidence, but which also enables posterior inference as a by-product, thereby allowing simultaneous parameter estimation and model selection. The widely-used MultiNest algorithm presents a particularly efficient implementation of the NS technique for multi-modal posteriors. In this paper we discuss importance nested sampling (INS), an alternative summation of the MultiNest draws, which can calculate the Bayesian evidence at up to an order of magnitude higher accuracy than `vanilla' NS with no change in the way MultiNest explores the parameter space. This is accomplished by treating as a (pseudo-)importance sample the totality of points collected by MultiNest, including those previously discarded under the constrained likelihood sampling of the NS algorithm. We apply this technique to several challenging test problems and compare the accuracy of Bayesian evidences obtained with INS against those from vanilla NS.
Metropolis nested sampling evolves a Markov chain from a current livepoint and accepts new points along the chain according to a version of the Metropolis acceptance ratio modified to satisfy the likelihood constraint, characteristic of nested sampling algorithms. The geometric nested sampling algorithm we present here is a based on the Metropolis method, but treats parameters as though they represent points on certain geometric objects, namely circles, tori and spheres. For parameters which represent points on a circle or torus, the trial distribution is `wrapped' around the domain of the pos…
Nested sampling is an increasingly popular technique for Bayesian computation, in particular for multimodal, degenerate problems of moderate to high dimensionality. Without appropriate settings, however, nested sampling software may fail to explore such posteriors correctly; for example producing correlated samples or missing important modes. This paper introduces new diagnostic tests to assess the reliability both of parameter estimation and evidence calculations using nested sampling software, and demonstrates them empirically. We present two new diagnostic plots for nested sampling, and giv…
In arXiv:0911.2150, Rutger van Haasteren seeks to criticize the nested sampling algorithm for Bayesian data analysis in general and its MultiNest implementation in particular. He introduces a new method for evidence evaluation based on the idea of Voronoi tessellation and requiring samples from the posterior distribution obtained through MCMC based methods. He compares its accuracy and efficiency with MultiNest, concluding that it outperforms MultiNest in several cases. This comparison is completely unfair since the proposed method can not perform the complete Bayesian data analysis including …
We introduce ScannerBit, the statistics and sampling module of the public, open-source global fitting framework GAMBIT. ScannerBit provides a standardised interface to different sampling algorithms, enabling the use and comparison of multiple computational methods for inferring profile likelihoods, Bayesian posteriors, and other statistical quantities. The current version offers random, grid, raster, nested sampling, differential evolution, Markov Chain Monte Carlo (MCMC) and ensemble Monte Carlo samplers. We also announce the release of a new standalone differential evolution sampler, Diver, …
In this paper we present an algorithm for rapid Bayesian analysis that combines the benefits of nested sampling and artificial neural networks. The blind accelerated multimodal Bayesian inference (BAMBI) algorithm implements the MultiNest package for nested sampling as well as the training of an artificial neural network (NN) to learn the likelihood function. In the case of computationally expensive likelihoods, this allows the substitution of a much more rapid approximation in order to increase significantly the speed of the analysis. We begin by demonstrating, with a few toy examples, the ab…
We describe an application of the MultiNest algorithm to gravitational wave data analysis. MultiNest is a multimodal nested sampling algorithm designed to efficiently evaluate the Bayesian evidence and return posterior probability densities for likelihood surfaces containing multiple secondary modes. The algorithm employs a set of live points which are updated by partitioning the set into multiple overlapping ellipsoids and sampling uniformly from within them. This set of live points climbs up the likelihood surface through nested iso-likelihood contours and the evidence and posterior distribu…
Since its debut by John Skilling in 2004, nested sampling has proven a valuable tool to the scientist, providing hypothesis evidence calculations and parameter inference for complicated posterior distributions, particularly in the field of astronomy. Due to its computational complexity and long-running nature, in the past, nested sampling has been reserved for offline-type Bayesian inference, leaving tools such as variational inference and MCMC for online-type, time-constrained, Bayesian computations. These tools do not easily handle complicated multi-modal posteriors, discrete random variable…
In this note, we shortly survey some recent approaches on the approximation of the Bayes factor used in Bayesian hypothesis testing and in Bayesian model choice. In particular, we reassess importance sampling, harmonic mean sampling, and nested sampling from a unified perspective.
One of the well-known challenges in optimal experimental design is how to efficiently estimate the nested integrations of the expected information gain. The Gaussian approximation and associated importance sampling have been shown to be effective at reducing the numerical costs. However, they may fail due to the non-negligible biases and the numerical instabilities. A new approach is developed to compute the expected information gain, when the posterior distribution is multimodal - a situation previously ignored by the methods aiming at accelerating the nested numerical integrations. Specifica…
Here we present an investigation into using nested sampling algorithms in cosmological likelihood analysis. We present a new nested sampling algorithm, ESNested, that uses Evolution Strategies for sample proposals. This quickly finds the maximum for complex likelihoods and provides an accurate measure of the Bayesian evidence, with no prior assumptions about the shape of the likelihood surface. We present the first cosmological constraints using Evolution Strategies, from WMAP 7, HST and SNIa data using likelihood and data provided with CosmoMC. We find a significantly higher maximum likelihoo…
In this paper we review the concepts of Bayesian evidence and Bayes factors, also known as log odds ratios, and their application to model selection. The theory is presented along with a discussion of analytic, approximate and numerical techniques. Specific attention is paid to the Laplace approximation, variational Bayes, importance sampling, thermodynamic integration, and nested sampling and its recent variants. Analogies to statistical physics, from which many of these techniques originate, are discussed in order to provide readers with deeper insights that may lead to new techniques. The u…
We propose a novel spike and slab prior specification with scaled beta prime marginals for the importance parameters of regression coefficients to allow for general effect selection within the class of structured additive distributional regression. This enables us to model effects on all distributional parameters for arbitrary parametric distributions, and to consider various effect types such as non-linear or spatial effects as well as hierarchical regression structures. Our spike and slab prior relies on a parameter expansion that separates blocks of regression coefficients into overall scal…
Recent cosmological analyses rely on the ability to accurately sample from high-dimensional posterior distributions. A variety of algorithms have been applied in the field, but justification of the particular sampler choice and settings is often lacking. Here we investigate three such samplers to motivate and validate the algorithm and settings used for the Dark Energy Survey (DES) analyses of the first 3 years (Y3) of data from combined measurements of weak lensing and galaxy clustering. We employ the full DES Year 1 likelihood alongside a much faster approximate likelihood, which enables us …
These notes aim at presenting an overview of Bayesian statistics, the underlying concepts and application methodology that will be useful to astronomers seeking to analyse and interpret a wide variety of data about the Universe. The level starts from elementary notions, without assuming any previous knowledge of statistical methods, and then progresses to more advanced, research-level topics. After an introduction to the importance of statistical inference for the physical sciences, elementary notions of probability theory and inference are introduced and explained. Bayesian methods are then p…
NAUTILUS: boosting Bayesian importance nested sampling with deep learning
Kavli Institute for Particle Astrophysics and Cosmology（卡夫利粒子天体物理与宇宙学研究所）；Stanford University（斯坦福大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；University of Michigan（密歇根大学）；Leinweber Center for Theoretical Physics（莱因韦伯理论物理中心）
We introduce a new Bayesian HI spectral line fitting technique capable of obtaining spectroscopic redshifts for millions of galaxies in radio surveys with the Square Kilometere Array (SKA). This technique is especially well-suited to the low signal-to-noise regime that the redshifted 21-cm HI emission line is expected to be observed in, especially with SKA Phase 1, allowing for robust source detection. After selecting a set of continuum objects relevant to large, cosmological-scale surveys with the first phase of the SKA dish array (SKA1-MID), we simulate data corresponding to their HI line em…
Most applications of Bayesian Inference for parameter estimation and model selection in astrophysics involve the use of Monte Carlo techniques such as Markov Chain Monte Carlo (MCMC) and nested sampling. However, these techniques are time consuming and their convergence to the posterior could be difficult to determine. In this work, we advocate Variational inference as an alternative to solve the above problems, and demonstrate its usefulness for parameter estimation and model selection in Astrophysics. Variational inference converts the inference problem into an optimization problem by approx…
Model fitting is possibly the most extended problem in science. Classical approaches include the use of least-squares fitting procedures and maximum likelihood methods to estimate the value of the parameters in the model. However, in recent years, Bayesian inference tools have gained traction. Usually, Markov chain Monte Carlo methods are applied to inference problems, but they present some disadvantages, particularly when comparing different models fitted to the same dataset. Other Bayesian methods can deal with this issue in a natural and effective way. We have implemented an importance samp…
We present results from a data challenge posed to the radial velocity (RV) community: namely, to quantify the Bayesian "evidence" for n={0,1,2,3} planets in a set of synthetically generated RV datasets containing a range of planet signals. Participating teams were provided the same likelihood function and set of priors to use in their analysis. They applied a variety of methods to estimate Z, the marginal likelihood for each n-planet model, including cross-validation, the Laplace approximation, importance sampling, and nested sampling. We found the dispersion in Z across different methods grew…
Simulation-based inference with neural posterior estimation applied to X-ray spectral fitting -- III Deriving exact posteriors with dimension reduction and importance sampling
Institut de Recherche en Astrophysique et Planétologie（天体物理学与行星学研究所以）
We investigate the utility to computational Bayesian analyses of a particular family of recursive marginal likelihood estimators characterized by the (equivalent) algorithms known as "biased sampling" or "reverse logistic regression" in the statistics literature and "the density of states" in physics. Through a pair of numerical examples (including mixture modeling of the well-known galaxy data set) we highlight the remarkable diversity of sampling schemes amenable to such recursive normalization, as well as the notable efficiency of the resulting pseudo-mixture distributions for gauging prior…
Flow Matching for Atmospheric Retrieval of Exoplanets: Where Reliability meets Adaptive Noise Levels
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；ETH Zurich（苏黎世联邦理工学院）；ELLIS Institute Tübingen（埃利斯研究所蒂宾根分支）；Max Planck Institute for Gravitational Physics (Albert Einstein Institute)（马克斯·普朗克引力物理研究所（阿尔伯特·爱因斯坦研究所））
