We are entering an era where progress in cosmology is driven by data, and alternative models will have to be compared and ruled out according to some consistent criterium. The most conservative and widely used approach is Bayesian model comparison. In this paper we explicitly calculate the Bayes factors for all models that are linear with respect to their parameters. We do this in order to test the so called Jeffreys' scale and determine analytically how accurate its predictions are in a simple case where we fully understand and can calculate everything analytically. We also discuss the case of nested models, e.g. one with $M_1$ and another with $M_2\supset M_1$ parameters and we derive analytic expressions for both the Bayes factor and the Figure of Merit, defined as the inverse area of the model parameter's confidence contours. With all this machinery and the use of an explicit example we demonstrate that the threshold nature of Jeffreys' scale is not a "one size fits all" reliable tool for model comparison and that it may lead to biased conclusions. Furthermore, we discuss the importance of choosing the right basis in the context of models that are linear with respect to their parameters and how that basis affects the parameter estimation and the derived constraints.
The Jeffreys prior for a constrained part of a parameter space is the same as that for the unconstrained space, contrary to the assertions of Hannestad and Tram.
Bayesian parameter inference depends on a choice of prior probability distribution for the parameters in question. The prior which makes the posterior distribution maximally sensitive to data is called the Jeffreys prior, and it is completely determined by the response of the likelihood to changes in parameters. Under the assumption that the likelihood is a Gaussian distribution, the Jeffreys prior is a constant, i.e. flat. However, if one parameter is constrained by physical considerations, the Gaussian approximation fails and the flat prior is no longer the Jeffreys prior. In this paper we c…
Large scale astronomical surveys are going wider and deeper than ever before. However, astronomers, cosmologists and theorists continue to face the perennial issue that their data sets are often incomplete in magnitude space and must be carefully treated in order to avoid Malmquist bias, especially in the field of supernova cosmology. Historically, cosmological parameter inference in supernova cosmology was done using $χ^2$ methodology; however, recent years have seen a rise in the use of Bayesian Hierarchical Models. In this paper we develop a Bayesian Hierarchical methodology to account for …
In this work we propose a new matrix-free implementation of the Wiener sampler which is traditionally applied to high dimensional analysis when signal covariances are unknown. Specifically, the proposed method addresses the problem of jointly inferring a high dimensional signal and its corresponding covariance matrix from a set of observations. Our method implements a Gibbs sampling adaptation of the previously presented messenger approach, permitting to cast the complex multivariate inference problem into a sequence of uni-variate random processes. In this fashion, the traditional requirement…
The standard Bayesian model formalism comparison cannot be applied to most cosmological models as they lack well-motivated parameter priors. However, if the data-set being used is separable then it is possible to use some of the data to obtain the necessary parameter distributions, the rest of the data being retained for model comparison. While such methods are not fully prescriptive, they provide a route to applying Bayesian model comparison in cosmological situations where it could not otherwise be used.
While recent supernova cosmology research has benefited from improved measurements, current analysis approaches are not statistically optimal and will prove insufficient for future surveys. This paper discusses the limitations of current supernova cosmological analyses in treating outliers, selection effects, shape- and color-standardization relations, unexplained dispersion, and heterogeneous observations. We present a new Bayesian framework, called UNITY (Unified Nonlinear Inference for Type-Ia cosmologY), that incorporates significant improvements in our ability to confront these effects. W…
We demonstrate the effectiveness of a Bayesian evidence-based analysis for diagnosing and disentangling the sky-averaged 21-cm signal from instrumental systematic effects. As a case study, we consider a simulated REACH pipeline with an injected systematic. We demonstrate that very poor performance or erroneous signal recovery is achieved if the systematic remains unmodelled. These effects include sky-averaged 21-cm posterior estimates resembling a very deep or wide signal. However, when including parameterised models of the systematic, the signal recovery is dramatically improved in performanc…
We propose here a \emph{novel} method which singles out the \emph{a priori} unavoidable dependence on the underlying cosmological model when extracting parameter constraints, providing robust limits which only depend on the considered dataset. Interestingly, when dealing with several possible cosmologies and interpreting the Bayesian preference in terms of the Gaussian statistical evidence, the preferred model is much less favored than when only two cases are compared. As a working example, we apply our approach to the cosmological neutrino mass bounds, which play a fundamental role not only i…
We look at the distribution of the Bayesian evidence for mock realizations of supernova and baryon acoustic oscillation data. The ratios of Bayesian evidences of different models are often used to perform model selection. The significance of these Bayes factors are then interpreted using scales such as the Jeffreys or Kass \& Raftery scale. First, we demonstrate how to use the evidence itself to validate the model, that is to say how well a model fits the data, regardless of how well other models perform. The basic idea is that if, for some real dataset a model's evidence lies outside the dist…
We present results exploring the role that probabilistic deep learning models can play in cosmology from large scale astronomical surveys through estimating the distances to galaxies (redshifts) from photometry. Due to the massive scale of data coming from these new and upcoming sky surveys, machine learning techniques using galaxy photometry are increasingly adopted to predict galactic redshifts which are important for inferring cosmological parameters such as the nature of dark energy. Associated uncertainty estimates are also critical measurements, however, common machine learning methods t…
pocoMC is a Python package for accelerated Bayesian inference in astronomy and cosmology. The code is designed to sample efficiently from posterior distributions with non-trivial geometry, including strong multimodality and non-linearity. To this end, pocoMC relies on the Preconditioned Monte Carlo algorithm which utilises a Normalising Flow in order to decorrelate the parameters of the posterior. It facilitates both tasks of parameter estimation and model comparison, focusing especially on computationally expensive applications. It allows fitting arbitrary models defined as a log-likelihood f…
Sparsity has been recently introduced in cosmology for weak-lensing and CMB data analysis for different applications such as denoising, component separation or inpainting (i.e. filling the missing data or the mask). Although it gives very nice numerical results, CMB sparse inpainting has been severely criticized by top researchers in cosmology, based on arguments derived from a Bayesian perspective. Trying to understand their point of view, we realize that interpreting a regularization penalty term as a prior in a Bayesian framework can lead to erroneous conclusions. This paper is by no means …
Simulation-based inference (SBI) is rapidly establishing itself as a standard machine learning technique for analyzing data in cosmological surveys. Despite continual improvements to the quality of density estimation by learned models, applications of such techniques to real data are entirely reliant on the generalization power of neural networks far outside the training distribution, which is mostly unconstrained. Due to the imperfections in scientist-created simulations, and the large computational expense of generating all possible parameter combinations, SBI methods in cosmology are vulner…
Recent technological advances have led to a flood of new data on cosmology rich in information about the formation and evolution of the universe, e.g., the data collected in Sloan Digital Sky Survey (SDSS) for more than 200 million objects. The analyses of such data demand cutting edge statistical technologies. Here, we have used the concept of mixture model within Bayesian semiparametric methodology to fit the regression curve with the bivariate data for the apparent magnitude and redshift for Quasars in SDSS (2007) catalogue. Associated with the mixture modeling is a highly efficient curve-f…
We consider some background tests of standard cosmology in the context of Horava gravity with different scaling dimensions for space and time, which has been proposed as a renormalizable, higher-derivative, Lorentz-violating quantum gravity model without ghost problems. We obtain the "very strong" and "strong" Bayesian evidence for our two cosmology models A and B, respectively, depending on the choice of parametrization based on Horava gravity, against the standard, spatially-flat, LCDM cosmology model based on general relativity. An MCMC analysis with observational data, including BAO, shows…
We review some of the common methods for model selection: the goodness of fit, the likelihood ratio test, Bayesian model selection using Bayes factors, and the classical as well as the Bayesian information theoretic approaches. We illustrate these different approaches by comparing models for the expansion history of the Universe. In the discussion we highlight the premises and objectives entering these different approaches to model selection and finally recommend the information theoretic approach.
We use Bayesian model selection techniques to test extensions of the standard flat LambdaCDM paradigm. Dark-energy and curvature scenarios, and primordial perturbation models are considered. To that end, we calculate the Bayesian evidence in favour of each model using Population Monte Carlo (PMC), a new adaptive sampling technique which was recently applied in a cosmological context. The Bayesian evidence is immediately available from the PMC sample used for parameter estimation without further computational effort, and it comes with an associated error evaluation. Besides, it provides an unbi…
These notes aim at presenting an overview of Bayesian statistics, the underlying concepts and application methodology that will be useful to astronomers seeking to analyse and interpret a wide variety of data about the Universe. The level starts from elementary notions, without assuming any previous knowledge of statistical methods, and then progresses to more advanced, research-level topics. After an introduction to the importance of statistical inference for the physical sciences, elementary notions of probability theory and inference are introduced and explained. Bayesian methods are then p…
Measurements of the equation of state of dark energy from surveys of thousands of Type Ia Supernovae (SNe Ia) will be limited by spectroscopic follow-up and must therefore rely on photometric identification, increasing the chance that the sample is contaminated by Core Collapse Supernovae (CC SNe). Bayesian methods for supernova cosmology can remove contamination bias while maintaining high statistical precision but are sensitive to the choice of parameterization of the contaminating distance distribution. We use simulations to investigate the form of the contaminating distribution and its dep…
Bayesian inference is often used in cosmology and astrophysics to derive constraints on model parameters from observations. This approach relies on the ability to compute the likelihood of the data given a choice of model parameters. In many practical situations, the likelihood function may however be unavailable or intractable due to non-gaussian errors, non-linear measurements processes, or complex data formats such as catalogs and maps. In these cases, the simulation of mock data sets can often be made through forward modeling. We discuss how Approximate Bayesian Computation (ABC) can be us…
Modern scientific cosmology pushes the boundaries of knowledge and the knowable. This is prompting questions on the nature of scientific knowledge. A central issue is what defines a 'good' model. When addressing global properties of the Universe or its initial state this becomes a particularly pressing issue. How to assess the probability of the Universe as a whole is empirically ambiguous, since we can examine only part of a single realisation of the system under investigation: at some point, data will run out. We review the basics of applying Bayesian statistical explanation to the Universe …
Theoretical and observational cosmology have enjoyed a number of significant successes over the last two decades. Cosmic microwave background measurements from the Wilkinson Microwave Anisotropy Probe and Planck, together with large-scale structure and supernova (SN) searches, have put very tight constraints on cosmological parameters. Type Ia supernovae (SNIa) played a central role in the discovery of the accelerated expansion of the Universe, recognised by the Nobel Prize in Physics in 2011. The last decade has seen an enormous increase in the amount of high quality SN observations, with SN …
We present the public release of the Bayesian sampling algorithm for cosmology, CosmoPMC (Cosmology Population Monte Carlo). CosmoPMC explores the parameter space of various cosmological probes, and also provides a robust estimate of the Bayesian evidence. CosmoPMC is based on an adaptive importance sampling method called Population Monte Carlo (PMC). Various cosmology likelihood modules are implemented, and new modules can be added easily. The importance-sampling algorithm is written in C, and fully parallelised using the Message Passing Interface (MPI). Due to very little overhead, the wall-…
There are several methods for model selection in cosmology which have at least two major goals, that of finding the correct model or predicting well. In this work we discuss through a study of well-known model selection methods like Akaike information criterion (AIC), Bayesian information criterion (BIC), deviance information criterion (DIC) and Bayesian evidence, how these different goals are pursued in each paradigm. We also apply another method for model selection which less seen in cosmological literature, the Cross-validation method. Using these methods we will compare two different scena…
Measures of discordance between datasets have become an essential part of cosmological analyses. It is important to accurately evaluate the significance of such discordances when present. We propose here a Bayesian interpretation of inconsistency measures that can extract information about physical inconsistencies in the presence of data scatter. The new framework is based on the conditional probability distribution of the level of physical inconsistency given the obtained value of the measure. We use the index of inconsistency as a case study to illustrate the new interpretation framework, bu…
Cosmological experiments often employ Bayesian workflows to derive constraints on cosmological and astrophysical parameters from their data. It has been shown that these constraints can be combined across different probes such as Planck and the Dark Energy Survey and that this can be a valuable exercise to improve our understanding of the universe and quantify tension between multiple experiments. However, these experiments are typically plagued by differing systematics, instrumental effects and contaminating signals, which we collectively refer to as `nuisance' components, that have to be mod…
The Lyman-alpha forest provides strong constraints on both cosmological parameters and intergalactic medium astrophysics, which are forecast to improve further with the next generation of surveys including eBOSS and DESI. As is generic in cosmological inference, extracting this information requires a likelihood to be computed throughout a high-dimensional parameter space. Evaluating the likelihood requires a robust and accurate mapping between the parameters and observables, in this case the 1D flux power spectrum. Cosmological simulations enable such a mapping, but due to computational time c…
Easy Parameter Inference in Cosmology (EPIC) is another Markov Chain Monte Carlo (MCMC) sampler for Cosmology. It is implemented in Python and provides Bayesian parameter inference and model comparison based on the Bayesian evidence. The Parallel Tempering algorithm is included, which can help in the exploration of posterior distributions with two or more separated peaks. Adaptive routines for obtaining better efficiency with fine-tuned algorithms are being developed and will be available in future versions. In this user's guide, I give general instructions for installation and usage, includin…
Towards end-to-end Bayesian forward models in global 21-cm cosmology: surrogate modelling and marginalisation of beam uncertainty
Astrophysics Group, Cavendish Laboratory（卡文迪许实验室天体物理组）；Kavli Institute for Cosmology（卡弗里宇宙学研究所）；Antenna Group, Université Catholique de Louvain（天主教鲁汶大学天线研究组）
We present a new Bayesian hierarchical model (BHM) named Steve for performing type Ia supernova (SNIa) cosmology fits. This advances previous works by including an improved treatment of Malmquist bias, accounting for additional sources of systematic uncertainty, and increasing numerical efficiency. Given light curve fit parameters, redshifts, and host-galaxy masses, we fit Steve simultaneously for parameters describing cosmology, SNIa populations, and systematic uncertainties. Selection effects are characterised using Monte-Carlo simulations. We demonstrate its implementation by fitting realis…
Current cosmological constraints on the scalar spectral index of primordial fluctuations $n_{\rm s}$ in the $Λ$CDM model have excluded the minimal scale-invariant Harrison-Zel'dovich model ($n_{\rm s}=1$; hereafter HZ) at high significance, providing support for inflation. In recent years, however, some tensions have emerged between different cosmological datasets that, if not due to systematics, could indicate the presence of new physics beyond the $Λ$CDM model. In the light of these developments, we evaluate the Bayesian evidence against HZ in different data combinations and model extensions…
Observational astrophysics consists of making inferences about the Universe by comparing data and models. The credible intervals placed on model parameters are often as important as the maximum a posteriori probability values, as the intervals indicate concordance or discordance between models and with measurements from other data. Intermediate statistics (e.g. the power spectrum) are usually measured and inferences made by fitting models to these rather than the raw data, assuming that the likelihood for these statistics has multivariate Gaussian form. The covariance matrix used to calculate …
It has been suggested that recent cosmological and flavor-oscillation data favor the existence of additional neutrino species beyond the three predicted by the Standard Model of particle physics. We apply Bayesian model selection to determine whether there is indeed any evidence from current cosmological datasets for the standard cosmological model to be extended to include additional neutrino flavors. The datasets employed include cosmic microwave background temperature, polarization and lensing power spectra, and measurements of the baryon acoustic oscillation scale and the Hubble constant. …
A hierarchical Bayesian framework for cosmology using Type 1 AGN variability
Institute of Astronomy, University of Cambridge（剑桥大学天文学研究所）；Kavli Institute for Cosmology, University of Cambridge（剑桥大学卡弗里宇宙学研究所）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge（剑桥大学应用数学与理论物理系）
In this paper, we use The Quijote simulations in order to extract the cosmological parameters through Bayesian Neural Networks. This kind of model has a remarkable ability to estimate the associated uncertainty, which is one of the ultimate goals in the precision cosmology era. We demonstrate the advantages of BNNs for extracting more complex output distributions and non-Gaussianities information from the simulations.
Bayesian Inference of Primordial Magnetic Field Parameters from CMB with Spherical Graph Neural Networks
Laboratorio de Inteligencia Artificial (SavIA Lab), Grupo Signos, Departamento de Matemáticas, Universidad El Bosque（人工智能实验室（SavIA实验室）、Signos小组、数学系、埃尔博斯克大学）
