Exponential random graph models are extremely difficult models to handle from a statistical viewpoint, since their normalising constant, which depends on model parameters, is available only in very trivial cases. We show how inference can be carried out in a Bayesian framework using a MCMC algorithm, which circumvents the need to calculate the normalising constants. We use a population MCMC approach which accelerates convergence and improves mixing of the Markov chain. This approach improves performance with respect to the Monte Carlo maximum likelihood method of Geyer and Thompson (1992).
In this paper, the panel count data analysis for recurrent events is considered. Such analysis is useful for studying tumor or infection recurrences in both clinical trial and observational studies. A bivariate Gaussian Cox process model is proposed to jointly model the observation process and the recurrent event process. Bayesian nonparametric inference is proposed for simultaneously estimating regression parameters, bivariate frailty effects and baseline intensity functions. Inference is done through Markov chain Monte Carlo, with fully developed computational techniques. Predictive inferenc…
In the last two decades, Bayesian inference has become commonplace in astronomy. At the same time, the choice of algorithms, terminology, notation, and interpretation of Bayesian inference varies from one sub-field of astronomy to the next, which can lead to confusion to both those learning and those familiar with Bayesian statistics. Moreover, the choice varies between the astronomy and statistics literature, too. In this paper, our goal is two-fold: (1) provide a reference that consolidates and clarifies terminology and notation across disciplines, and (2) outline practical guidance for Baye…
Analysis of competing risks data plays an important role in the lifetime data analysis. Recently Feizjavadian and Hashemi (Computational Statistics and Data Analysis, vol. 82, 19-34, 2015) provided a classical inference of a competing risks data set using four-parameter Marshall-Olkin bivariate Weibull distribution when the failure of an unit at a particular time point can happen due to more than one cause. The aim of this paper is to provide the Bayesian analysis of the same model based on a very flexible Gamma-Dirichlet prior on the scale parameters. It is observed that the Bayesian inferenc…
We present a new probabilistic method for detecting, deblending, and cataloging astronomical sources called the Bayesian Light Source Separator (BLISS). BLISS is based on deep generative models, which embed neural networks within a Bayesian model. For posterior inference, BLISS uses a new form of variational inference known as Forward Amortized Variational Inference. The BLISS inference routine is fast, requiring a single forward pass of the encoder networks on a GPU once the encoder networks are trained. BLISS can perform fully Bayesian inference on megapixel images in seconds, and produces h…
This paper presents a Bayesian optimization method with exponential convergence without the need of auxiliary optimization and without the delta-cover sampling. Most Bayesian optimization methods require auxiliary optimization: an additional non-convex global optimization problem, which can be time-consuming and hard to implement in practice. Also, the existing Bayesian optimization method with exponential convergence requires access to the delta-cover sampling, which was considered to be impractical. Our approach eliminates both requirements and achieves an exponential convergence rate.
Recently, a very attractive logistic regression inference method for exponential family Gibbs spatial point processes was introduced. We combined it with the technique of quadratic tangential variational approximation and derived a new Bayesian technique for analysing spatial point patterns. The technique is described in detail, and demonstrated on numerical examples.
We combine Bayesian prediction and weighted inference as a unified approach to survey inference. The general principles of Bayesian analysis imply that models for survey outcomes should be conditional on all variables that affect the probability of inclusion. We incorporate the weighting variables under the framework of multilevel regression and poststratification, as a byproduct generating model-based weights after smoothing. We investigate deep interactions and introduce structured prior distributions for smoothing and stability of estimates. The computation is done via Stan and implemented …
Bayesian synthetic likelihood (BSL) is now an established method for conducting approximate Bayesian inference in models where, due to the intractability of the likelihood function, exact Bayesian approaches are either infeasible or computationally too demanding. Implicit in the application of BSL is the assumption that the data generating process (DGP) can produce simulated summary statistics that capture the behaviour of the observed summary statistics. We demonstrate that if this compatibility between the actual and assumed DGP is not satisfied, i.e., if the model is misspecified, BSL can y…
Both necessary and sufficient conditions for Bayesian exponential consistency
在过去的十年中，贝叶斯非参数过程的渐近理论有了显著的发展。指数一致性在这方面发挥了重要作用。已知$f_0$在先验的Kullback-Leibler支持下的条件不能保证后验的指数一致性。许多作者得到了后验的指数相合性的额外充分条件，例如，参见Schwartz（1965），Barron，Schervish and Wasserman（1999），Ghosal，Ghosh and Ramamoorthi（1999），Walker（2004），Xing and Ranneby（2008）。然而，由于Kullback-Leibler支持条件，很少知道的充要条件和充分条件。本文给出了一类既有充要条件。作为结果，我们得到了一个简单的贝叶斯指数一致性的充分条件，这是弱于以前的充分条件。
We formulate Bayesian updates in Markov processes by means of path integral techniques and derive the imaginary-time Schrödinger equation with likelihood to direct the inference incorporated as a potential for the posterior probability distribution
We introduce the concepts of Bayesian lens, characterizing the bidirectional structure of exact Bayesian inference, and statistical game, formalizing the optimization objectives of approximate inference problems. We prove that Bayesian inversions compose according to the compositional lens pattern, and exemplify statistical games with a number of classic statistical concepts, from maximum likelihood estimation to generalized variational Bayesian methods. This paper is the first in a series laying the foundations for a compositional account of the theory of active inference, and we therefore pa…
We investigate the geometry of the maximal a posteriori (MAP) partition in the Bayesian Mixture Model where the component and the base distributions are chosen from conjugate exponential families. We prove that in this case the clusters are separated by the contour lines of a linear functional of the sufficient statistic. As a particular example, we describe Bayesian Mixture of Normals with Normal-inverse-Wishart prior on the component mean and covariance, in which the clusters in any MAP partition are separated by a quadratic surface. In connection with results of Rajkowski (2018), where the …
The notion of Craig interpolant, used as a form of explanation in automated reasoning, is adapted from logical inference to statistical inference and used to explain inferences made by neural networks. The method produces explanations that are at the same time concise, understandable and precise.
Fitting models to data using Bayesian inference is quite common, but when each point in parameter space gives a curve, fitting the curve to a data set requires new nuisance parameters, which specify the metric embedding the one-dimensional curve into the higher-dimensional space occupied by the data. A generic formalism for curve fitting in the context of Bayesian inference is developed which shows how the aforementioned metric arises. The result is a natural generalization of previous works, and is compared to oft-used frequentist approaches and similar Bayesian techniques.
College of Information and Computer Sciences（信息与计算机科学学院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
The study of private inference has been sparked by growing concern regarding the analysis of data when it stems from sensitive sources. We present the first method for private Bayesian inference in exponential families that properly accounts for noise introduced by the privacy mechanism. It is efficient because it works only with sufficient statistics and not individual data. Unlike other methods, it gives properly calibrated posterior beliefs in the non-asymptotic data regime.
We introduce new estimation methods for a sub-class of the Gaussian scale mixture models for wavelet trees by Wainwright, Simoncelli & Willsky that rely on modern results for composite likelihoods and approximate Bayesian inference. Our methodology is illustrated for denoising and edge detection problems in two-dimensional images.
We discuss Bayesian inference for parameters selected using the data. First, we provide a critical analysis of the existing positions in the literature regarding the correct Bayesian approach under selection. Second, we propose two types of non-informative priors for selection models. These priors may be employed to produce a posterior distribution in the absence of prior information as well as to provide well-calibrated frequentist inference for the selected parameter. We test the proposed priors empirically in several scenarios.
We propose a quasi-Bayesian method to conduct inference for the production frontier. This approach combines multiple first-stage extreme quantile estimates by the quasi-Bayesian method to produce the point estimate and confidence interval for the production frontier. We show the asymptotic properties of the proposed estimator and the validity of the inference procedure. The finite sample performance of our method is illustrated through simulations and an empirical application.
Machine learning provides algorithms that can learn from data and make inferences or predictions on data. Bayesian networks are a class of graphical models that allow to represent a collection of random variables and their condititional dependencies by directed acyclic graphs. In this paper, an inference algorithm for the hidden random variables of a Bayesian network is given by using the tropicalization of the marginal distribution of the observed variables. By restricting the topological structure to graded networks, an inference algorithm for graded Bayesian networks will be established tha…
In this paper, we derive a Bayesian model order selection rule by using the exponentially embedded family method, termed Bayesian EEF. Unlike many other Bayesian model selection methods, the Bayesian EEF can use vague proper priors and improper noninformative priors to be objective in the elicitation of parameter priors. Moreover, the penalty term of the rule is shown to be the sum of half of the parameter dimension and the estimated mutual information between parameter and observed data. This helps to reveal the EEF mechanism in selecting model orders and may provide new insights into the ope…
Typical Bayesian inference requires parameter identification via likelihood parameterization, which has invited criticism for being less flexible than the Frequentist framework and subject to misspecification. Though misspecification may be avoided by functional parameter inference under a nonparametric model space, there does not exist a flexible Bayesian semiparametric model that would allow full control over the marginal prior over any general functional parameter. We present the technique of $θ$-augmentation which helps us manipulate nonparametric models into semiparametric ones that direc…
Inspired by the pioneering work of Rubin (1978), we employ the potential outcomes framework to develop a finite-population Bayesian causal inference framework for randomized controlled $2^K$ factorial designs with binary outcomes, which are common in medical research. As demonstrated by simulated and empirical examples, the proposed framework corrects the well-known variance over-estimation issue of the classic "Neymanian" inference framework, under various settings.
Modern deep neural network models suffer from adversarial examples, i.e. confidently misclassified points in the input space. It has been shown that Bayesian neural networks are a promising approach for detecting adversarial points, but careful analysis is problematic due to the complexity of these models. Recently Gilmer et al. (2018) introduced adversarial spheres, a toy set-up that simplifies both practical and theoretical analysis of the problem. In this work, we use the adversarial sphere set-up to understand the properties of approximate Bayesian inference methods for a linear model in a…
We revisit and generalize the concept of composite likelihood as a method to make a probabilistic inference by aggregation of multiple Bayesian agents, thereby defining a class of predictive models which we call composite Bayesian. This perspective gives insight to choose the weights associated with composite likelihood, either a priori or via learning; in the latter case, they may be tuned so as to minimize prediction cross-entropy, yielding an easy-to-solve convex problem. We argue that composite Bayesian inference is a middle way between generative and discriminative models that trades off …
We consider exponential two-armed bandit problem in which incomes are described by exponential distribution densities. We develop Bayesian approach and present recursive equation for determination of Bayesian strategy and Bayesian risk. In the limiting case as the control horizon goes to infinity, we obtain the second order partial differential equation in the domain of "close distributions". Results are compared with Gaussian two-armed bandit. It turned out that exponential and Gaussian two-armed bandits have the same description in the limiting case. Since Gaussian two-armed bandit describes…
The exponential and the Lomax distributions are widely used in life testing experiments in mixture models. A mixture model of exponential distribution and Lomax distribution is proposed. Parameters of the proposed model are estimated using classical and Bayesian procedures under type-I right censoring. Expressions for Bayes estimators are derived assuming noninformative (uniform and Jeffreys) priors under symmetric and asymmetric loss functions. Posterior predictive distributions of a future observation are derived and predictive estimates are obtained. Extensive Monte Carlo simulations are ca…
This paper provides a review of Approximate Bayesian Computation (ABC) methods for carrying out Bayesian posterior inference, through the lens of density estimation. We describe several recent algorithms and make connection with traditional approaches. We show advantages and limitations of models based on parametric approaches and we then draw attention to developments in machine learning, which we believe have the potential to make ABC scalable to higher dimensions and may be the future direction for research in this area.
The Bergm package provides a comprehensive framework for Bayesian inference using Markov chain Monte Carlo (MCMC) algorithms. It can also supply graphical Bayesian goodness-of-fit procedures that address the issue of model adequacy. The package is simple to use and represents an attractive way of analysing network data as it offers the advantage of a complete probabilistic treatment of uncertainty. Bergm is based on the ergm package and therefore it makes use of the same model set-up and network simulation algorithms. The Bergm package has been continually improved in terms of speed performanc…
Bayesian, frequentist and fiducial (BFF) inferences are much more congruous than they have been perceived historically in the scientific community (cf., Reid and Cox 2015; Kass 2011; Efron 1998). Most practitioners are probably more familiar with the two dominant statistical inferential paradigms, Bayesian inference and frequentist inference. The third, lesser known fiducial inference paradigm was pioneered by R.A. Fisher in an attempt to define an inversion procedure for inference as an alternative to Bayes' theorem. Although each paradigm has its own strengths and limitations subject to thei…
In this paper, a scale mixture of Normal distributions model is developed for classification and clustering of data having outliers and missing values. The classification method, based on a mixture model, focuses on the introduction of latent variables that gives us the possibility to handle sensitivity of model to outliers and to allow a less restrictive modelling of missing data. Inference is processed through a Variational Bayesian Approximation and a Bayesian treatment is adopted for model learning, supervised classification and clustering.
University of Cambridge（剑桥大学）；IT University of Copenhagen（哥本哈根信息技术大学）
In this paper, we study the trade-offs of different inference approaches for Bayesian matrix factorisation methods, which are commonly used for predicting missing values, and for finding patterns in the data. In particular, we consider Bayesian nonnegative variants of matrix factorisation and tri-factorisation, and compare non-probabilistic inference, Gibbs sampling, variational Bayesian inference, and a maximum-a-posteriori approach. The variational approach is new for the Bayesian nonnegative models. We compare their convergence, and robustness to noise and sparsity of the data, on both synt…
In this article we consider Bayesian inference for partially observed Andersson-Madigan-Perlman (AMP) Gaussian chain graph (CG) models. Such models are of particular interest in applications such as biological networks and financial time series. The model itself features a variety of constraints which make both prior modeling and computational inference challenging. We develop a framework for the aforementioned challenges, using a sequential Monte Carlo (SMC) method for statistical inference. Our approach is illustrated on both simulated data as well as real case studies from university gradua…
