4 pages, 2 figures; to appear in Statistical Challenges in 21st Century Cosmology, Proceedings IAU Symposium No. 306, A. H. Heavens & J.-L. Starck, eds
The standard Bayesian model formalism comparison cannot be applied to most cosmological models as they lack well-motivated parameter priors. However, if the data-set being used is separable then it is possible to use some of the data to obtain the necessary parameter distributions, the rest of the data being retained for model comparison. While such methods are not fully prescriptive, they provide a route to applying Bayesian model comparison in cosmological situations where it could not otherwise be used.
These are the proceedings of the First Astrostatistics School: Bayesian Methods in Cosmology, held in Bogotá D.C., Colombia, June 9-13, 2014. The first astrostatistics school has been the first event in Colombia where statisticians and cosmologists from some universities in Bogotá met to discuss the statistic methods applied to cosmology, especially the use of Bayesian statistics in the study of Cosmic Microwave Background (CMB), Baryonic Acoustic Oscillations (BAO), Large Scale Structure (LSS) and weak lensing.
These notes aim at presenting an overview of Bayesian statistics, the underlying concepts and application methodology that will be useful to astronomers seeking to analyse and interpret a wide variety of data about the Universe. The level starts from elementary notions, without assuming any previous knowledge of statistical methods, and then progresses to more advanced, research-level topics. After an introduction to the importance of statistical inference for the physical sciences, elementary notions of probability theory and inference are introduced and explained. Bayesian methods are then p…
The problem of testing mutually exclusive hypotheses with dependent test statistics is considered. Bayesian and frequentist approaches to multiplicity control are studied and compared to help gain understanding as to the effect of test statistic dependence on each approach. The Bayesian approach is shown to have excellent frequentist properties and is argued to be the most effective way of obtaining frequentist multiplicity control, without sacrificing power, when there is considerable test statistic dependence.
In this paper, we adopt a Bayesian point of view for predicting real continuous-time processes. We give two equivalent definitions of a Bayesian predictor and study some properties: admissibility, prediction sufficiency, non-unbiasedness, comparison with efficient predictors. Prediction of Poisson process and prediction of Ornstein-Uhlenbeck process in the continuous and sampled situations are considered. Various simulations illustrate comparison with non-Bayesian predictors.
This paper revisits the procedures suggested by Dudewicz and Dalal (1975) and Rinott (1978) which are designed for selecting the population with the highest mean among independent Gaussian populations with unknown and possibly different variances. In a previous paper Jacobovic and Zuk (2017) made a conjecture that the relative asymptotic efficiency of these procedures equals to the ratio of two certain sequences. This work suggests a quasi-Bayesian modelling of the problem under which this conjecture is valid. In addition, this paper motivates an open question regarding the extreme value distr…
The main object of Bayesian statistical inference is the determination of posterior distributions. Sometimes these laws are given for quantities devoid of empirical value. This serious drawback vanishes when one confines oneself to considering a finite horizon framework. However, assuming infinite exchangeability gives rise to fairly tractable {\it a posteriori} quantities, which is very attractive in applications. Hence, with a view to a reconciliation between these two aspects of the Bayesian way of reasoning, in this paper we provide quantitative comparisons between posterior distributions …
Mixture models are well-known for their versatility, and the Bayesian paradigm is a suitable platform for mixture analysis, particularly when the number of components is unknown. Bhattacharya (2008) introduced a mixture model based on the Dirichlet process, where an upper bound on the unknown number of components is to be specified. Here we consider a Bayesian asymptotic framework for objectively specifying the upper bound, which we assume to depend on the sample size. In particular, we define a Bayesian analogue of the mean integrated squared error (Bayesian MISE), and select that form of the…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both parameter estimation and model-order selection. Extending existing results for multinomial models of discrete data, we connect inference to statistical mechanics through information-theoretic (type theory) techniques. We establish a direct relationship between Bayesian evidence and the partition function which allows for straightforward calculation of the expectati…
In this paper, we use The Quijote simulations in order to extract the cosmological parameters through Bayesian Neural Networks. This kind of model has a remarkable ability to estimate the associated uncertainty, which is one of the ultimate goals in the precision cosmology era. We demonstrate the advantages of BNNs for extracting more complex output distributions and non-Gaussianities information from the simulations.
This contribution presents a Bayesian approach to the issue of linking of the results from key comparison measurements. A mathematical treatment based on Bayesian statistics for the analysis of the results from two comparisons with some joint participants is described. This robust statistical analysis provides expressions and standard uncertainties for the key comparison reference value (KCRV) and the degree of equivalence (DOE) as well as a conformity check without any assumption on a priority of one of the comparisons. In addition to the derivation of the mathematical formulae to be used for…
A paired comparison analysis is the simplest way to make comparative judgments between objects where objects may be goods, services or skills. For a set of problems, this technique helps to choose the most important problem to solve first and/or provides the solution that will be the most effective. This paper presents the theory of paired comparisons method and contributes to the paired comparisons models by developing a new model based on t-distribution. The developed model is illustrated using a data set of citations among four famous journals of Statistics. Using Bayesian analysis, the jou…
Two procedures for checking Bayesian models are compared using a simple test problem based on the local Hubble expansion. Over four orders of magnitude, p-values derived from a global goodness-of-fit criterion for posterior probability density functions (Lucy 2017) agree closely with posterior predictive p-values. The former can therefore serve as an effective proxy for the difficult-to-calculate posterior predictive p-values.
Wrocław University of Science and Technology（弗罗茨瓦夫科技大学）；City University of New York（纽约城市大学）
Recent work raises concerns about the use of standard splits to compare natural language processing models. We propose a Bayesian statistical model comparison technique which uses k-fold cross-validation across multiple data sets to estimate the likelihood that one model will outperform the other, or that the two will produce practically equivalent results. We use this technique to rank six English part-of-speech taggers across two data sets and three evaluation metrics.
Bayesian model comparison relies upon the model evidence, yet for many models of interest the model evidence is unavailable in closed form and must be approximated. Many of the estimators for evidence that have been proposed in the Monte Carlo literature suffer from high variability. This paper considers the reduction of variance that can be achieved by exploiting control variates in this setting. Our methodology is based on thermodynamic integration and applies whenever the gradient of both the log-likelihood and the log-prior with respect to the parameters can be efficiently evaluated. Resul…
This article introduces the bpcs R package (Bayesian Paired Comparison in Stan) and the statistical models implemented in the package. This package aims to facilitate the use of Bayesian models for paired comparison data in behavioral research. Bayesian analysis of paired comparison data allows parameter estimation even in conditions where the maximum likelihood does not exist, allows easy extension of paired comparison models, provide straightforward interpretation of the results with credible intervals, have better control of type I error, have more robust evidence towards the null hypothesi…
We consider the problems of hypothesis testing and model comparison under a flexible Bayesian linear regression model whose formulation is closely connected with the linear mixed effect model and the parametric models for SNP set analysis in genetic association studies. We derive a class of analytic approximate Bayes factors and illustrate their connections with a variety of frequentist test statistics, including the Wald statistic and the variance component score statistic. Taking advantage of Bayesian model averaging and hierarchical modeling, we demonstrate some distinct advantages and flex…
Bayesian approaches for handling covariate measurement error are well established, and yet arguably are still relatively little used by researchers. For some this is likely due to unfamiliarity or disagreement with the Bayesian inferential paradigm. For others a contributory factor is the inability of standard statistical packages to perform such Bayesian analyses. In this paper we first give an overview of the Bayesian approach to handling covariate measurement error, and contrast it with regression calibration (RC), arguably the most commonly adopted approach. We then argue why the Bayesian …
In social and biomedical sciences testing in contingency tables often involves order restrictions on cell-probabilities parameters. We develop objective Bayes methods for order-constrained testing and model comparison when observations arise under product binomial or multinomial sampling. Specifically, we consider tests for monotone order of the parameters against equality of all parameters. Our strategy combines in a unified way both the intrinsic prior methodology and the encompassing prior approach in order to compute Bayes factors and posterior model probabilities. Performance of our metho…
We propose a novel Bayesian model selection technique on linear mixed-effects models to compare multiple treatments with a control. A fully Bayesian approach is implemented to estimate the marginal inclusion probabilities that provide a direct measure of the difference between treatments and the control, along with the model-averaged posterior distributions. Default priors are proposed for model selection incorporating domain knowledge and a component-wise Gibbs sampler is developed for efficient posterior computation. We demonstrate the proposed method based on simulated data and an experimen…
Within path sampling framework, we show that probability distribution divergences, such as the Chernoff information, can be estimated via thermodynamic integration. The Boltzmann-Gibbs distribution pertaining to different Hamiltonians is implemented to derive tempered transitions along the path, linking the distributions of interest at the endpoints. Under this perspective, a geometric approach is feasible, which prompts intuition and facilitates tuning the error sources. Additionally, there are direct applications in Bayesian model evaluation. Existing marginal likelihood and Bayes factor est…
University of Stuttgart（斯图加特大学）；Heidelberg University（海德堡大学）
Bayesian model comparison (BMC) offers a principled probabilistic approach to study and rank competing models. In standard BMC, we construct a discrete probability distribution over the set of possible models, conditional on the observed data of interest. These posterior model probabilities (PMPs) are measures of uncertainty, but -- when derived from a finite number of observations -- are also uncertain themselves. In this paper, we conceptualize distinct levels of uncertainty which arise in BMC. We explore a fully probabilistic framework for quantifying meta-uncertainty, resulting in an appli…
In this paper, we propose a nonparametric Bayesian approach for Lindsey and penalized Gaussian mixtures methods. We compare these methods with the Dirichlet process mixture model. Our approach is a Bayesian nonparametric method not based solely on a parametric family of probability distributions. Thus, the fitted models are more robust to model misspecification. Also, with the Bayesian approach, we have the entire posterior distribution of our parameter of interest; it can be summarized through credible intervals, mean, median, standard deviation, quantiles, etc. The Lindsey, penalized Gaussia…
Bayesian models of legal arguments generally aim to produce a single integrated model, combining each of the legal arguments under consideration. This combined approach implicitly assumes that variables and their relationships can be represented without any contradiction or misalignment, and in a way that makes sense with respect to the competing argument narratives. This paper describes a novel approach to compare and 'average' Bayesian models of legal arguments that have been built independently and with no attempt to make them consistent in terms of variables, causal assumptions or parametr…
Recent advances in deep reinforcement learning have made significant strides in performance on applications such as Go and Atari games. However, developing practical methods to balance exploration and exploitation in complex domains remains largely unsolved. Thompson Sampling and its extension to reinforcement learning provide an elegant approach to exploration that only requires access to posterior samples of the model. At the same time, advances in approximate Bayesian methods have made posterior approximation for flexible neural network models practical. Thus, it is attractive to consider a…
Imperial College London（伦敦帝国学院）；Sciemus Ltd（Sciemus有限公司）；University of Oxford（牛津大学）
In a recent paper [1] we introduced the Fuzzy Bayesian Learning (FBL) paradigm where expert opinions can be encoded in the form of fuzzy rule bases and the hyper-parameters of the fuzzy sets can be learned from data using a Bayesian approach. The present paper extends this work for selecting the most appropriate rule base among a set of competing alternatives, which best explains the data, by calculating the model evidence or marginal likelihood. We explain why this is an attractive alternative over simply minimizing a mean squared error metric of prediction and show the validity of the propos…
Circular data are data measured in angles and occur in a variety of scientific disciplines. Bayesian methods promise to allow for flexible analysis of circular data. Three existing MCMC methods (Gibbs, Metropolis-Hastings, and Rejection) for a single group of circular data were extended to be used in a between-subjects design, providing a novel procedure to compare groups of circular data. Investigating the performance of the methods by simulation study, all methods were found to overestimate the concentration parameter of the posterior, while coverage was reasonable. The rejection sampler per…
Istituto Dalle Molle di studi sull’Intelligenza Artificiale (IDSIA)（达勒莫勒人工智能研究所 (IDSIA)）；University of Ljubljana（卢布尔雅那大学）
Usually one compares the accuracy of two competing classifiers via null hypothesis significance tests (nhst). Yet the nhst tests suffer from important shortcomings, which can be overcome by switching to Bayesian hypothesis testing. We propose a Bayesian hierarchical model which jointly analyzes the cross-validation results obtained by two classifiers on multiple data sets. It returns the posterior probability of the accuracies of the two classifiers being practically equivalent or significantly different. A further strength of the hierarchical model is that, by jointly analyzing the results ob…
Comment on Vogels et al.'s (2024) "Bayesian Structure Learning in Undirected Gaussian Graphical Models: Literature Review with Empirical Comparison": An updated performance check using BGGM
Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO subsampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orde…
We consider the problem of statistical inference on unknown quantities structured as a multiway table. We show that such multiway tables are naturally formed by arranging regression coefficients in complex systems of linear models for association analysis. In genetics and genomics, the resulting two-way and three-way tables cover many important applications. Within the Bayesian hierarchical model framework, we define the structure of a multiway table through prior specification. Focusing on model comparison and selection, we derive analytic expressions of Bayes factors and their approximations…
The aim of this work is the comparison of the performance of the three popular software platforms JAGS, NIMBLE and Stan. These probabilistic programming languages are able to automatically generate samples from the posterior distribution of interest using MCMC algorithms, starting from the specification of a Bayesian model, i.e. the likelihood and the prior. The final goal is to present a detailed analysis of their strengths and weaknesses to statisticians or applied scientists. In this way, we wish to contribute to make them fully aware of the pros and cons of this software. We carry out a sy…
We are interested in learning how listeners perceive sounds as having human origins. An experiment was performed with a series of electronically synthesized sounds, and listeners were asked to compare them in pairs. We propose a Bayesian probabilistic method to learn individual preferences from non-transitive pairwise comparison data, as happens when one (or more) individual preferences in the data contradicts what is implied by the others. We build a Bayesian Mallows model in order to handle non-transitive data, with a latent layer of uncertainty which captures the generation of preference mi…
Université de Lyon（里昂大学）；CNRS（法国国家科学研究中心）；Université Lyon 1（里昂第一大学）
We present a novel hybrid algorithm for Bayesian network structure learning, called Hybrid HPC (H2PC). It first reconstructs the skeleton of a Bayesian network and then performs a Bayesian-scoring greedy hill-climbing search to orient the edges. It is based on a subroutine called HPC, that combines ideas from incremental and divide-and-conquer constraint-based methods to learn the parents and children of a target variable. We conduct an experimental comparison of H2PC against Max-Min Hill-Climbing (MMHC), which is currently the most powerful state-of-the-art algorithm for Bayesian network stru…
Objective: To provide guidance for the use of the main functions available in R for performing post hoc Bayesian analysis of a randomized clinical trial with a survival endpoint using proportional hazard models. Study Design and Setting: Data derived from the ALLOZITHRO trial, conducted with 465 patients after allograft to prevent pulmonary complications and allocated between azithromycin and placebo; airflow decline-free survival at 2 years after randomization was the main endpoint. Results: Despite heterogeneity in modeling assumptions, in particular for the baseline hazard (parametric or no…
Network meta-analysis is a powerful tool to synthesize evidence from independent studies and compare multiple treatments simultaneously. A critical task of performing a network meta-analysis is to offer ranks of all available treatment options for a specific disease outcome. Frequently, the estimated treatment rankings are accompanied by a large amount of uncertainty, suffer from multiplicity issues, and rarely permit ties. These issues make interpreting rankings problematic as they are often treated as absolute metrics. To address these shortcomings, we formulate a ranking strategy that adapt…
In this paper, we compare the performance of two methods for estimating Bayesian networks from data containing exogenous variables and random effects. The first method is fully Bayesian in which a prior distribution is placed on the exogenous variables, whereas the second method, which we call the residual approach, accounts for the effects of exogenous variables by using the notion of restricted maximum likelihood. We review the two score-based metrics, then study their performance by measuring the Kullback Leibler divergence, or distance, between the two resulting posterior density functions…
Geochemical processes in subsurface reservoirs affected by microbial activity change the material properties of porous media. This is a complex biogeochemical process in subsurface reservoirs that currently contains strong conceptual uncertainty. This means, several modeling approaches describing the biogeochemical process are plausible and modelers face the uncertainty of choosing the most appropriate one. Once observation data becomes available, a rigorous Bayesian model selection accompanied by a Bayesian model justifiability analysis could be employed to choose the most appropriate model, …
Large scale astronomical surveys are going wider and deeper than ever before. However, astronomers, cosmologists and theorists continue to face the perennial issue that their data sets are often incomplete in magnitude space and must be carefully treated in order to avoid Malmquist bias, especially in the field of supernova cosmology. Historically, cosmological parameter inference in supernova cosmology was done using $χ^2$ methodology; however, recent years have seen a rise in the use of Bayesian Hierarchical Models. In this paper we develop a Bayesian Hierarchical methodology to account for …
While recent supernova cosmology research has benefited from improved measurements, current analysis approaches are not statistically optimal and will prove insufficient for future surveys. This paper discusses the limitations of current supernova cosmological analyses in treating outliers, selection effects, shape- and color-standardization relations, unexplained dispersion, and heterogeneous observations. We present a new Bayesian framework, called UNITY (Unified Nonlinear Inference for Type-Ia cosmologY), that incorporates significant improvements in our ability to confront these effects. W…
We demonstrate the effectiveness of a Bayesian evidence-based analysis for diagnosing and disentangling the sky-averaged 21-cm signal from instrumental systematic effects. As a case study, we consider a simulated REACH pipeline with an injected systematic. We demonstrate that very poor performance or erroneous signal recovery is achieved if the systematic remains unmodelled. These effects include sky-averaged 21-cm posterior estimates resembling a very deep or wide signal. However, when including parameterised models of the systematic, the signal recovery is dramatically improved in performanc…
We present results exploring the role that probabilistic deep learning models can play in cosmology from large scale astronomical surveys through estimating the distances to galaxies (redshifts) from photometry. Due to the massive scale of data coming from these new and upcoming sky surveys, machine learning techniques using galaxy photometry are increasingly adopted to predict galactic redshifts which are important for inferring cosmological parameters such as the nature of dark energy. Associated uncertainty estimates are also critical measurements, however, common machine learning methods t…
pocoMC is a Python package for accelerated Bayesian inference in astronomy and cosmology. The code is designed to sample efficiently from posterior distributions with non-trivial geometry, including strong multimodality and non-linearity. To this end, pocoMC relies on the Preconditioned Monte Carlo algorithm which utilises a Normalising Flow in order to decorrelate the parameters of the posterior. It facilitates both tasks of parameter estimation and model comparison, focusing especially on computationally expensive applications. It allows fitting arbitrary models defined as a log-likelihood f…
Simulation-based inference (SBI) is rapidly establishing itself as a standard machine learning technique for analyzing data in cosmological surveys. Despite continual improvements to the quality of density estimation by learned models, applications of such techniques to real data are entirely reliant on the generalization power of neural networks far outside the training distribution, which is mostly unconstrained. Due to the imperfections in scientist-created simulations, and the large computational expense of generating all possible parameter combinations, SBI methods in cosmology are vulner…
We consider some background tests of standard cosmology in the context of Horava gravity with different scaling dimensions for space and time, which has been proposed as a renormalizable, higher-derivative, Lorentz-violating quantum gravity model without ghost problems. We obtain the "very strong" and "strong" Bayesian evidence for our two cosmology models A and B, respectively, depending on the choice of parametrization based on Horava gravity, against the standard, spatially-flat, LCDM cosmology model based on general relativity. An MCMC analysis with observational data, including BAO, shows…
Recent technological advances have led to a flood of new data on cosmology rich in information about the formation and evolution of the universe, e.g., the data collected in Sloan Digital Sky Survey (SDSS) for more than 200 million objects. The analyses of such data demand cutting edge statistical technologies. Here, we have used the concept of mixture model within Bayesian semiparametric methodology to fit the regression curve with the bivariate data for the apparent magnitude and redshift for Quasars in SDSS (2007) catalogue. Associated with the mixture modeling is a highly efficient curve-f…
We review some of the common methods for model selection: the goodness of fit, the likelihood ratio test, Bayesian model selection using Bayes factors, and the classical as well as the Bayesian information theoretic approaches. We illustrate these different approaches by comparing models for the expansion history of the Universe. In the discussion we highlight the premises and objectives entering these different approaches to model selection and finally recommend the information theoretic approach.
We propose here a \emph{novel} method which singles out the \emph{a priori} unavoidable dependence on the underlying cosmological model when extracting parameter constraints, providing robust limits which only depend on the considered dataset. Interestingly, when dealing with several possible cosmologies and interpreting the Bayesian preference in terms of the Gaussian statistical evidence, the preferred model is much less favored than when only two cases are compared. As a working example, we apply our approach to the cosmological neutrino mass bounds, which play a fundamental role not only i…
