Scientific analyses often rely on slow, but accurate forward models for observable data conditioned on known model parameters. While various emulation schemes exist to approximate these slow calculations, these approaches are only safe if the approximations are well understood and controlled. This workshop submission reviews and updates a previously published method, which has been used in cosmological simulations, to (1) train an emulator while simultaneously estimating posterior probabilities with MCMC and (2) explicitly propagate the emulation error into errors on the posterior probabilities for model parameters. We demonstrate how these techniques can be applied to quickly estimate posterior distributions for parameters of the $Λ$CDM cosmology model, while also gauging the robustness of the emulator approximation.
When researchers carry out a null hypothesis significance test, it is tempting to assume that a statistically significant result lowers Prob(H0), the probability of the null hypothesis being true. Technically, such a statement is meaningless for various reasons: e.g., the null hypothesis does not have a probability associated with it. However, it is possible to relax certain assumptions to compute the posterior probability Prob(H0) under repeated sampling. We show in a step-by-step guide that the intuitively appealing belief, that Prob(H0) is low when significant results have been obtained und…
Emulation of complex computer simulations have become an effective tool in the exploration of the behaviour of the simulated processes. Agriculture is one such area where the simulation of crop growth, nutrition, soil condition and pollution could be invaluable in any land management decisions. In this paper, we study output from the EPIC simulation model to investigate the behaviour of crop yield in response to changes in inputs such as fertilizer levels, soil, steepness, and other environmental covariates. We build a model for crop yield around a non-linear Mitscherlich Baule growth model to…
National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校国家超级计算应用中心）
We introduce deep learning models to estimate the masses of the binary components of black hole mergers, $(m_1,m_2)$, and three astrophysical properties of the post-merger compact remnant, namely, the final spin, $a_f$, and the frequency and damping time of the ringdown oscillations of the fundamental $\ell=m=2$ bar mode, $(ω_R, ω_I)$. Our neural networks combine a modified $\texttt{WaveNet}$ architecture with contrastive learning and normalizing flow. We validate these models against a Gaussian conjugate prior family whose posterior distribution is described by a closed analytical expression.…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；University of Seoul（首尔市立大学）；EWHA Womans University（梨花女子大学）
A simulation is useful when the phenomenon of interest is either expensive to regenerate or irreproducible with the same context. Recently, Bayesian inference on the distribution of the simulation input parameter has been implemented sequentially to minimize the required simulation budget for the task of simulation validation to the real-world. However, the Bayesian inference is still challenging when the ground-truth posterior is multi-modal with a high-dimensional simulation output. This paper introduces a regularization technique, namely Neural Posterior Regularization (NPR), which enforces…
IIt is known that a statistically significant treatment may not be clinically significant. A quantity that can be used to assess clinical significance is called the minimum clinically important difference (MCID), and inference on the MCID is an important and challenging problem. Modeling for the purpose of inference on the MCID is non-trivial, and concerns about bias from a misspecified parametric model or inefficiency from a nonparametric model motivate an alternative approach to balance robustness and efficiency. In particular, a recently proposed representation of the MCID as the minimizer …
We discuss the Bayesian emulation approach to computational solution of multi-step portfolio studies in financial time series. "Bayesian emulation for decisions" involves mapping the technical structure of a decision analysis problem to that of Bayesian inference in a purely synthetic "emulating" statistical model. This provides access to standard posterior analytic, simulation and optimization methods that yield indirect solutions of the decision problem. We develop this in time series portfolio analysis using classes of economically and psychologically relevant multi-step ahead portfolio uti…
Synthetic control methods have gained popularity among causal studies with observational data, particularly when estimating the impacts of the interventions that are implemented to a small number of large units. Implementing the synthetic control methods faces two major challenges: a) estimating weights for each control unit to create a synthetic control and b) providing statistical inferences. To overcome these challenges, we propose a Bayesian framework that implements the synthetic control method with the parallelly shiftable convex hull and provides a useful Bayesian inference, which is dr…
In frequentist inference, minimizing the Hellinger distance between a kernel density estimate and a parametric family produces estimators that are both robust to outliers and statistically efficienty when the parametric model is correct. This paper seeks to extend these results to the use of nonparametric Bayesian density estimators within disparity methods. We propose two estimators: one replaces the kernel density estimator with the expected posterior density from a random histogram prior; the other induces a posterior over parameters through the posterior for the random histogram. We show t…
Estimating the marginal likelihoods is an essential feature of model selection in the Bayesian context. It is especially crucial to have good estimates when assessing the number of planets orbiting stars when the models explain the noisy data with different numbers of Keplerian signals. We introduce a simple method for approximating the marginal likelihoods in practice when a statistically representative sample from the parameter posterior density is available. We use our truncated posterior mixture estimate to receive accurate model probabilities for models with differing number of Keplerian …
University of California, Berkeley（加州大学伯克利分校）；Intel Corporation（英特尔公司）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel version of Celeste, suitable for learning catalogs from modern large-scale astronomical datasets. Our algorithmic innovations include a fast numerical optimization routine for Bayesian posterior inference and a statistically efficient scheme for decomposing astronomical optimizati…
Aalto University（阿尔托大学）；University of Helsinki（赫尔辛基大学）；University of Manchester（曼彻斯特大学）
Generalized linear models (GLMs) such as logistic regression are among the most widely used arms in data analyst's repertoire and often used on sensitive datasets. A large body of prior works that investigate GLMs under differential privacy (DP) constraints provide only private point estimates of the regression coefficients, and are not able to quantify parameter uncertainty. In this work, with logistic and Poisson regression as running examples, we introduce a generic noise-aware DP Bayesian inference method for a GLM at hand, given a noisy sum of summary statistics. Quantifying uncertainty a…
We find empirical evidence that mean-reverting jump processes are not statistically adequate to model electricity spot price spikes but independent, signed sums of such processes are statistically adequate. Further we demonstrate a change in the composition of these sums after a major economic event. This is achieved by developing a Markov Chain Monte Carlo (MCMC) procedure for Bayesian model calibration and a Bayesian assessment of model adequacy (posterior predictive checking). In particular we determine the number of signed mean-reverting jump components required in the APXUK and EEX market…
We address uncertainty quantification for Gaussian processes (GPs) under misspecified priors, with an eye towards Bayesian Optimization (BO). GPs are widely used in BO because they easily enable exploration based on posterior uncertainty bands. However, this convenience comes at the cost of robustness: a typical function encountered in practice is unlikely to have been drawn from the data scientist's prior, in which case uncertainty estimates can be misleading, and the resulting exploration can be suboptimal. We present a frequentist approach to GP/BO uncertainty quantification. We utilize the…
NTT DATA Mathematical Systems Inc.（NTT数据数学系统公司）；Tokyo Institute of Technology（东京工业大学）
Non-negative matrix factorization (NMF) is a knowledge discovery method that is used in many fields. Variational inference and Gibbs sampling methods for it are also wellknown. However, the variational approximation error has not been clarified yet, because NMF is not statistically regular and the prior distribution used in variational Bayesian NMF (VBNMF) has zero or divergence points. In this paper, using algebraic geometrical methods, we theoretically analyze the difference in negative log evidence (a.k.a. free energy) between VBNMF and Bayesian NMF, i.e., the Kullback-Leibler divergence be…
Performing exact posterior inference in complex generative models is often difficult or impossible due to an expensive to evaluate or intractable likelihood function. Approximate Bayesian computation (ABC) is an inference framework that constructs an approximation to the true likelihood based on the similarity between the observed and simulated data as measured by a predefined set of summary statistics. Although the choice of appropriate problem-specific summary statistics crucially influences the quality of the likelihood approximation and hence also the quality of the posterior sample in ABC…
We present $\it{CosmoPower}$, a suite of neural cosmological power spectrum emulators providing orders-of-magnitude acceleration for parameter estimation from two-point statistics analyses of Large-Scale Structure (LSS) and Cosmic Microwave Background (CMB) surveys. The emulators replace the computation of matter and CMB power spectra from Boltzmann codes; thus, they do not need to be re-trained for different choices of astrophysical nuisance parameters or redshift distributions. The matter power spectrum emulation error is less than $0.4\%$ in the wavenumber range $k \in [10^{-5}, 10] \, \mat…
Inference on the minimum clinically important difference, or MCID, is an important practical problem in medicine. The basic idea is that a treatment being statistically significant may not lead to an improvement in the patients' well-being. The MCID is defined as a threshold such that, if a diagnostic measure exceeds this threshold, then the patients are more likely to notice an improvement. Typical formulations use an underspecified model, which makes a genuine Bayesian solution out of reach. Here, for a challenging personalized MCID problem, where the practically-significant threshold depend…
Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation
Department of Computer and Data Sciences, Case Western Reserve University（计算机与数据科学系，凯斯西储大学）；Department of Physics, Case Western Reserve University（物理系，凯斯西储大学）
In this work the issue of Bayesian inference for stationary data is addressed. Therefor a parametrization of a statistically suitable subspace of the the shift-ergodic probability measures on a Cartesian product of some finite state space is given using an inverse limit construction. Moreover, an explicit model for the prior is given by taking into account an additional step in the usual stepwise sampling scheme of data. An update to the posterior is defined by exploiting this augmented sample scheme. Thereby, its model-step is updated using a measurement of the empirical distances between the…
Integrative analyses based on statistically relevant associations between genomics and a wealth of intermediary phenotypes (such as imaging) provide vital insights into their clinical relevance in terms of the disease mechanisms. Estimates for uncertainty in the resulting integrative models are however unreliable unless inference accounts for the selection of these associations with accuracy. In this article, we develop selection-aware Bayesian methods which: (i) counteract the impact of model selection bias through a "selection-aware posterior" in a flexible class of integrative Bayesian mode…
Surrogate models have become ubiquitous in science and engineering for their capability of emulating expensive computer codes, necessary to model and investigate complex phenomena. Bayesian emulators based on Gaussian processes adequately quantify the uncertainty that results from the cost of the original simulator, and thus the inability to evaluate it on the whole input space. However, it is common in the literature that only a partial Bayesian analysis is carried out, whereby the underlying hyper-parameters are estimated via gradient-free optimisation or genetic algorithms, to name a few me…
We develop a Bayesian Land Surface Phenology (LSP) model and examine its performance using Enhanced Vegetation Index (EVI) observations derived from the Harmonized Landsat Sentinel-2 (HLS) dataset. Building on previous work, we propose a double logistic function that, once couched within a Bayesian model, yields posterior distributions for all LSP parameters. We assess the efficacy of the Normal, Truncated Normal, and Beta likelihoods to deliver robust LSP parameter estimates. Two case studies are presented and used to explore aspects of the proposed model. The first, conducted over forested p…
The ensemble Kalman inversion (EKI) for the solution of Bayesian inverse problems of type $y = A u +\varepsilon$, with $u$ being an unknown parameter, $y$ a given datum, and $\varepsilon$ measurement noise, is a powerful tool usually derived from a sequential Monte Carlo point of view. It describes the dynamics of an ensemble of particles $\{u^j(t)\}_{j=1}^J$, whose initial empirical measure is sampled from the prior, evolving over an artificial time $t$ towards an approximate solution of the inverse problem, with $t=1$ emulating the posterior, and $t\to\infty$ corresponding to the under-regul…
How should social scientists understand and communicate the uncertainty of statistically estimated causal effects? I propose we utilize the posterior distribution of a causal effect and present the probability of the effect being greater (in absolute terms) than different minimum effect sizes. Probability is an intuitive measure of uncertainty for understanding and communication. In addition, the proposed approach needs no decision threshold for an uncertainty measure or an effect size, unlike the conventional approaches. I apply the proposed approach to a previous social scientific study, sho…
The Dirichlet process mixture (DPM) is a ubiquitous, flexible Bayesian nonparametric statistical model. However, full probabilistic inference in this model is analytically intractable, so that computationally intensive techniques such as Gibb's sampling are required. As a result, DPM-based methods, which have considerable potential, are restricted to applications in which computational resources and time for inference is plentiful. For example, they would not be practical for digital signal processing on embedded hardware, where computational resources are at a serious premium. Here, we develo…
This paper studies the numerical computation of integrals, representing estimates or predictions, over the output $f(x)$ of a computational model with respect to a distribution $p(\mathrm{d}x)$ over uncertain inputs $x$ to the model. For the functional cardiac models that motivate this work, neither $f$ nor $p$ possess a closed-form expression and evaluation of either requires $\approx$ 100 CPU hours, precluding standard numerical integration methods. Our proposal is to treat integration as an estimation problem, with a joint model for both the a priori unknown function $f$ and the a priori un…
Kavli Institute for Particle Astrophysics and Cosmology（卡弗里粒子天体物理与宇宙学研究所）；Stanford University（斯坦福大学）；SLAC National Accelerator Laboratory（SLAC国家加速器实验室）
In the past few years, approximate Bayesian Neural Networks (BNNs) have demonstrated the ability to produce statistically consistent posteriors on a wide range of inference problems at unprecedented speed and scale. However, any disconnect between training sets and the distribution of real-world objects can introduce bias when BNNs are applied to data. This is a common challenge in astrophysics and cosmology, where the unknown distribution of objects in our Universe is often the science goal. In this work, we incorporate BNNs with flexible posterior parameterizations into a hierarchical infere…
Simulation-based inference with neural posterior estimation applied to X-ray spectral fitting -- III Deriving exact posteriors with dimension reduction and importance sampling
Leveraging rapid parameter estimates for efficient gravitational-wave Bayesian inference via posterior repartitioning
Kavli Institute for Cosmology（卡弗里宇宙学研究所）；Cavendish Laboratory（卡文迪许实验室）；Institute of Cosmology and Gravitation, University of Portsmouth（朴茨茅斯大学宇宙学与引力研究所）
Bayesian component separation and power spectrum estimation for 21 cm intensity mapping data cubes
University of Western Cape（西开普敦大学）；Jodrell Bank Centre for Astrophysics, University of Manchester（曼彻斯特大学乔德雷克天体物理学中心）；South African Radio Astronomy Observatory（南非射电天文台）；Institute of Cosmology and Gravitation, University of Portsmouth（朴茨茅斯宇宙学与引力研究所）
Our aim is to present a fast and general Bayesian inference framework based on the synergy between machine learning techniques and standard sampling methods and apply it to infer the physical properties of clumpy dusty torus using infrared photometric high spatial resolution observations of active galactic nuclei. We make use of the Metropolis-Hastings Markov Chain Monte Carlo algorithm for sampling the posterior distribution function. Such distribution results from combining all a-priori knowledge about the parameters of the model and the information introduced by the observations. The main d…
Because many of our X-ray telescopes are optimized towards observing faint sources, observations of bright sources like X-ray binaries in outburst are often affected by instrumental biases. These effects include dead time and photon pile-up, which can dramatically change the statistical inference of physical parameters from these observations. While dead time is difficult to take into account in a statistically consistent manner, simulating dead time-affected data is often straightforward. This structure makes the issue of inferring physical properties from dead time-affected observations fall…
The weak-field approximation is one of the simplest models that allows us to relate the observed polarization induced by the Zeeman effect with the magnetic field vector present on the plasma of interest. It is usually applied for diagnosing magnetic fields in the solar and stellar atmospheres. A fully Bayesian approach to the inference of magnetic properties in unresolved structures is presented. The analytical expression for the marginal posterior distribution is obtained, from which we can obtain statistically relevant information about the model parameters. The role of a-priori information…
