Estimating a joint Highest Posterior Density credible set for a multivariate posterior density is challenging as dimension gets larger. Credible intervals for univariate marginals are usually presented for ease of computation and visualisation. There are often two layers of approximation, as we may need to compute a credible set for a target density which is itself only an approximation to the true posterior density. We obtain joint Highest Posterior Density credible sets for density estimation trees given by Li et al. (2016) approximating a density truncated to a compact subset of R^d as this is preferred to a copula construction. These trees approximate a joint posterior distribution from posterior samples using a piecewise constant function defined by sequential binary splits. We use a consistent estimator to measure of the symmetric difference between our credible set estimate and the true HPD set of the target density samples. This quality measure can be computed without the need to know the true set. We show how the true-posterior-coverage of an approximate credible set estimated for an approximate target density may be estimated in doubly intractable cases where posterior samples are not available. We illustrate our methods with simulation studies and find that our estimator is competitive with existing methods.
I begin my discussion by giving an overview of the main results. Then I proceed to touch upon issues about whether the credible ball constructed can be interpreted as a confidence ball, suggestions on reducing computational costs, and posterior consistency or contraction rates.
We present a discussion of the paper "Bayesian cluster analysis: point estimation and credible balls" by Wade and Ghahramani. We believe that this paper contributes substantially to the literature on Bayesian clustering by filling in an important methodological gap, by providing a means to assess the uncertainty around a point estimate of the optimal clustering solution based on a given loss function. In our discussion we reflect on the characterisation of uncertainty around the Bayesian optimal partition, revealing other possible alternatives that may be viable. In addition, we suggest other …
Clustering is widely studied in statistics and machine learning, with applications in a variety of fields. As opposed to classical algorithms which return a single clustering solution, Bayesian nonparametric models provide a posterior over the entire space of partitions, allowing one to assess statistical properties, such as uncertainty on the number of clusters. However, an important problem is how to summarize the posterior; the huge dimension of partition space and difficulties in visualizing it add to this problem. In a Bayesian analysis, the posterior of a real-valued parameter of interes…
Frequentist coverage of $(1-α)$-highest posterior density (HPD) credible sets is studied in a signal plus noise model under a large class of noise distributions. We consider a specific class of spike-and-slab prior distributions. Different regimes are identified and we derive closed form expressions for the $(1-α)$-HPD on each of these regimes. Similar to the earlier work by Marchand and Strawderman, it is shown that under suitable conditions, the frequentist coverage can drop to $1-3α/2.$
We study the problem of estimating the mode and maximum of an unknown regression function in the presence of noise. We adopt the Bayesian approach by using tensor-product B-splines and endowing the coefficients with Gaussian priors. In the usual fixed-in-advanced sampling plan, we establish posterior contraction rates for mode and maximum and show that they coincide with the minimax rates for this problem. To quantify estimation uncertainty, we construct credible sets for these two quantities that have high coverage probabilities with optimal sizes. If one is allowed to collect data sequential…
We investigate the frequentist coverage properties of Bayesian credible sets in a general, adaptive, nonparametric framework. It is well known that the construction of adaptive and honest confidence sets is not possible in general. To overcome this problem we introduce an extra assumption on the functional parameters, the so called "general polished tail" condition. We then show that under standard assumptions both the hierarchical and empirical Bayes methods results in honest confidence sets for sieve type of priors in general settings and we characterize their size. We apply the derived abst…
Model-assisted estimation with complex survey data is an important practical problem in survey sampling. When there are many auxiliary variables, selecting significant variables associated with the study variable would be necessary to achieve efficient estimation of population parameters of interest. In this paper, we formulate a regularized regression estimator in the framework of Bayesian inference using the penalty function as the shrinkage prior for model selection. The proposed Bayesian approach enables us to get not only efficient point estimates but also reasonable credible intervals. R…
Online experiments are a fundamental component of the development of web-facing products. Given their large user-bases, even small product improvements can have a large impact on user engagement or profits on an absolute scale. As a result, accurately estimating the relative impact of these changes is extremely important. I propose an approach based on an objective Bayesian model to improve the sensitivity of percent change estimation in A/B experiments. Leveraging pre-period information, this approach produces more robust and accurate point estimates and up to 50% tighter credible intervals t…
In this paper we consider Bayesian estimation for the parameters of inverse Gaussian distribution. Our emphasis is on Markov Chain Monte Carlo methods. We provide complete implementation of the Gibbs sampler algorithm. Assuming an informative prior, Bayes estimates are computed using the output of the Gibbs sampler and also from Lindley's approximation method. Maximum Likelihood and Uniformly Minimum Variance Unbiased estimates are obtained as well. We also compute Highest Posterior Density credible intervals, exact confidence intervals as well as "percentile" and "percentile-t" bootstrap appr…
CP4SBI: Local Conformal Calibration of Credible Sets in Simulation-Based Inference
Department of Statistics, Federal University of São Carlos（统计系，圣卡洛斯联邦大学）；Institute of Mathematics and Computer Science, University of São Paulo（数学与计算机科学学院，圣保罗大学）；Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK（格勒诺布尔阿尔卑斯大学，法国国家信息与自动化研究所，法国国家科学研究中心，格勒诺布尔INP，LJK）
In this paper, we propose a nonparametric Bayesian approach for Lindsey and penalized Gaussian mixtures methods. We compare these methods with the Dirichlet process mixture model. Our approach is a Bayesian nonparametric method not based solely on a parametric family of probability distributions. Thus, the fitted models are more robust to model misspecification. Also, with the Bayesian approach, we have the entire posterior distribution of our parameter of interest; it can be summarized through credible intervals, mean, median, standard deviation, quantiles, etc. The Lindsey, penalized Gaussia…
We propose a novel Dirichlet-based Pólya tree (D-P tree) prior on the copula and based on the D-P tree prior, a nonparametric Bayesian inference procedure. Through theoretical analysis and simulations, we are able to show that the flexibility of the D-P tree prior ensures its consistency in copula estimation, thus able to detect more subtle and complex copula structures than earlier nonparametric Bayesian models, such as a Gaussian copula mixture. Further, the continuity of the imposed D-P tree prior leads to a more favorable smoothing effect in copula estimation over classic frequentist metho…
In this paper, we investigate the classical and Bayesian estimation of unknown parameters of the Gumbel type-II distribution based on adaptive type-II progressive hybrid censored sample (AT-II PHCS). The maximum likelihood estimates (MLEs) and maximum product spacing estimates (MPSEs) are developed and computed numerically using Newton-Raphson method. Bayesian approaches are employed to estimate parameters under symmetric and asymmetric loss functions. Bayesian estimates are not in explicit forms. Thus, Bayesian estimates are obtained by using Markov chain Monte Carlo (MCMC) method along with …
Given a sample of size $n$ from a population of individuals belonging to different species with unknown proportions, a popular problem of practical interest consists in making inference on the probability $D_{n}(l)$ that the $(n+1)$-th draw coincides with a species with frequency $l$ in the sample, for any $l=0,1,\ldots,n$. This paper contributes to the methodology of Bayesian nonparametric inference for $D_{n}(l)$. Specifically, under the general framework of Gibbs-type priors we show how to derive credible intervals for a Bayesian nonparametric estimation of $D_{n}(l)$, and we investigate th…
For a general statistical model, we introduce the notion of data dependent measure (DDM) on the model parameter. Typical examples of DDM are the posterior distributions. Like for posteriors, the quality of a DDM is characterized by the contraction rate which we allow to be local, i.e., depending on the parameter. We construct confidence sets as DDM-credible sets and address the issue of optimality of such sets, via a trade-off between its "size" (the local radial rate) and its coverage probability. In the mildly ill-posed inverse signal-in-white-noise model, we construct a DDM as empirical Bay…
There has been an intense development of Bayes graphical model estimation approaches over the past decade - however, most of the existing methods are restricted to moderate dimensions. We propose a novel approach suitable for high dimensional settings, by decoupling model fitting and covariance selection. First, a full model based on a complete graph is fit under novel class of continuous shrinkage priors on the precision matrix elements, which induces shrinkage under an equivalence with Cholesky-based regularization while enabling conjugate updates of entire precision matrices. Subsequently, …
City University of Hong Kong（香港城市大学）；Arizona State University（亚利桑那州立大学）
This paper develops a novel stochastic tree ensemble method for nonlinear regression, which we refer to as XBART, short for Accelerated Bayesian Additive Regression Trees. By combining regularization and stochastic search strategies from Bayesian modeling with computationally efficient techniques from recursive partitioning approaches, the new method attains state-of-the-art performance: in many settings it is both faster and more accurate than the widely-used XGBoost algorithm. Via careful simulation studies, we demonstrate that our new approach provides accurate point-wise estimates of the m…
Nonresponse weighting adjustment using the response propensity score is a popular tool for handling unit nonresponse. Statistical inference after the nonresponse weighting adjustment is complicated because the effect of estimating the propensity model parameter needs to be incorporated. In this paper, we propose an approximate Bayesian approach to handle unit nonresponse with parametric model assumptions on the response probability, but without model assumptions for the outcome variable. The proposed Bayesian method is calibrated to the frequentist inference in that the credible region obtaine…
We consider statistical inference in the density estimation model using a tree-based Bayesian approach, with Optional Pólya trees as prior distribution. We derive near-optimal convergence rates for corresponding posterior distributions with respect to the supremum norm. For broad classes of Hölder-smooth densities, we show that the method automatically adapts to the unknown Hölder regularity parameter. We consider the question of uncertainty quantification by providing mathematical guarantees for credible sets from the obtained posterior distributions, leading to near-optimal uncertainty quant…
While the analysis of airborne laser scanning (ALS) data often provides reliable estimates for certain forest stand attributes -- such as total volume or basal area -- there is still room for improvement, especially in estimating species-specific attributes. Moreover, while information on the estimate uncertainty would be useful in various economic and environmental analyses on forests, a computationally feasible framework for uncertainty quantifying in ALS is still missing. In this article, the species-specific stand attribute estimation and uncertainty quantification (UQ) is approached using…
In the density estimation model, the question of adaptive inference using Pólya tree-type prior distributions is considered. A class of prior densities having a tree structure, called spike-and-slab Pólya trees, is introduced. For this class, two types of results are obtained: first, the Bayesian posterior distribution is shown to converge at the minimax rate for the supremum norm in an adaptive way, for any Hölder regularity of the true density between $0$ and $1$, thereby providing adaptive counterparts to the results for classical Pólya trees in Castillo (2017). Second, the question of unce…
The paper addresses the problem of estimation of the model parameters of the logistic exponential distribution based on progressive type-I hybrid censored sample. The maximum likelihood estimates are obtained and computed numerically using Newton-Raphson method. Further, the Bayes estimates are derived under squared error, LINEX and generalized entropy loss functions. Two types (independent and bivariate) of prior distributions are considered for the purpose of Bayesian estimation. It is seen that the Bayes estimates are not of explicit forms.Thus, Lindley's approximation technique is employed…
We explain how effective automatic probability density function estimates can be constructed using contemporary Bayesian inference engines such as those based on no-U-turn sampling and expectation propagation. Extensive simulation studies demonstrate that the proposed density estimates have excellent comparative performance and scale well to very large sample sizes due to a binning strategy. Moreover, the approach is fully Bayesian and all estimates are accompanied by pointwise credible intervals. An accompanying package in the R language facilitates easy use of the new density estimates.
Quantifying uncertainty of individualized treatment effects in right-censored survival data: A comparison of Bayesian additive regression trees and causal survival forest
Usual estimation methods for the parameters of extreme values distribution employ only a few values, wasting a lot of information. More precisely, in the case of the Gumbel distribution, only the block maxima values are used. In this work, we propose a method to seize all the available information in order to increase the accuracy of the estimations. This intent can be achieved by taking advantage of the existing relationship between the parameters of the baseline distribution, which generates data from the full sample space, and the ones for the limit Gumbel distribution. In this way, an info…
Uncertainty quantification is a critical missing component in radio interferometric imaging that will only become increasingly important as the big-data era of radio interferometry emerges. Statistical sampling approaches to perform Bayesian inference, like Markov Chain Monte Carlo (MCMC) sampling, can in principle recover the full posterior distribution of the image, from which uncertainties can then be quantified. However, for massive data sizes, like those anticipated from the Square Kilometre Array (SKA), it will be difficult if not impossible to apply any MCMC technique due to its inheren…
In a general class of Bayesian nonparametric models, we prove that the posterior distribution can be asymptotically approximated by a Gaussian process. Our results apply to nonparametric exponential family that contains both Gaussian and non-Gaussian regression, and also hold for both efficient (root-n) and inefficient (non root-n) estimation. Our general approximation theorem does not rely on posterior conjugacy, and can be verified in a class of Gaussian process priors that has a smoothing spline interpretation [59, 44]. In particular, the limiting posterior measure becomes prior-free under …
Probability estimation is one of the fundamental tasks in statistics and machine learning. However, standard methods for probability estimation on discrete objects do not handle object structure in a satisfactory manner. In this paper, we derive a general Bayesian network formulation for probability estimation on leaf-labeled trees that enables flexible approximations which can generalize beyond observations. We show that efficient algorithms for learning Bayesian networks can be easily extended to probability estimation on this challenging structured space. Experiments on both synthetic and r…
This article presents a new method for estimating the amount of an artifact class in use at a given moment in the past from a random assemblage of archaeological finds. This method is based on the use of simulation, since an analytical solution is computationally impractical. Estimating the number of artifacts in use at any time $t$ is shown to follow a Poisson distribution, which allows for credible intervals to be established using the Jeffreys prior. This estimator works from minimal assumptions about the dating and duration of finds, as well as the intensity of collection, and is applied t…
We investigate the problem of constructing Bayesian credible sets that are honest and adaptive for the L2-loss over a scale of Sobolev classes with regularity ranging between [D; 2D], for some given D in the context of the signal-in-white-noise model. We consider a scale of prior distributions indexed by a regularity hyper-parameter and choose the hyper-parameter both by marginal likelihood empirical Bayes and by hierarchical Bayes method, respectively. Next we consider a ball centered around the corresponding posterior mean with prescribed posterior probability. We show by theory and examples…
The hybrid censoring is a mixture of Type I and Type II censoring schemes. This paper presents the statistical inferences of the Inverse Weibull distribution when the data are Type-I hybrid censored. First we consider the maximum likelihood estimators of the unknown parameters. It is observed that the maximum likelihood estimators can not be obtained in closed form. We further obtain the Bayes estimators and the corresponding highest posterior density credible intervals of the unknown parameters under the assumption of independent gamma priors using the importance sampling procedure. We also c…
Sequential model-based optimization sequentially selects a candidate point by constructing a surrogate model with the history of evaluations, to solve a black-box optimization problem. Gaussian process (GP) regression is a popular choice as a surrogate model, because of its capability of calculating prediction uncertainty analytically. On the other hand, an ensemble of randomized trees is another option and has practical merits over GPs due to its scalability and easiness of handling continuous/discrete mixed variables. In this paper we revisit various ensembles of randomized trees to investig…
Institute of Statistical Mathematics（统计数理研究所）；The Graduate University for Advanced Science (SOKENDAI)（综合研究大学院大学）
Group testing is a method of identifying infected patients by performing tests on a pool of specimens collected from patients. For the case in which the test returns a false result with finite probability, we propose Bayesian inference and a corresponding belief propagation (BP) algorithm to identify the infected patients from the results of tests performed on the pool. We show that the true-positive rate is improved by taking into account the credible interval of a point estimate of each patient. Further, the prevalence and the error probability in the test are estimated by combining an expec…
We revisit the Bayesian Context Trees (BCT) modelling framework for discrete time series, which was recently found to be very effective in numerous tasks including model selection, estimation and prediction. A novel representation of the induced posterior distribution on model space is derived in terms of a simple branching process, and several consequences of this are explored in theory and in practice. First, it is shown that the branching process representation leads to a simple variable-dimensional Monte Carlo sampler for the joint posterior distribution on models and parameters, which can…
Traditionally, phylogeny and sequence alignment are estimated separately: first estimate a multiple sequence alignment and then infer a phylogeny based on the sequence alignment estimated in the previous step. However, uncertainty in the alignment estimation is ignored, resulting, possibly, in overstated certainty in phylogeny estimates. We develop a joint model for co-estimating phylogeny and sequence alignment which improves estimates from the traditional approach by accounting for uncertainty in the alignment in phylogenetic inferences. Our insertion and deletion (indel) model allows arbitr…
The present paper considers modified extension of the exponential distribution with three parameters. We study the main properties of this new distribution, with special emphasis on its median, mode and moments function and some characteristics related to reliability studies. For Modified- extension exponential distribution (MEXED) we have obtained the Bayes Estimators of scale and shape parameters using Lindley's approximation (L-approximation) under squared error loss function. But, through this approximation technique it is not possible to compute the interval estimates of the parameters. T…
The analysis of progressively censored data has received considerable attention in the last few years. In this paper we consider the joint progressive censoring scheme for two populations. It is assumed that the lifetime distribution of the items from the two populations follow Weibull distribution with the same shape but different scale parameters. Based on the joint progressive censoring scheme first we consider the maximum likelihood estimators of the unknown parameters whenever they exist. We provide the Bayesian inferences of the unknown parameters under a fairly general priors on the sha…
One of the features inherent in nested Archimedean copulas, also called hierarchical Archimedean copulas, is their rooted tree structure. A nonparametric, rank-based method to estimate this structure is presented. The idea is to represent the target structure as a set of trivariate structures, each of which can be estimated individually with ease. Indeed, for any three variables there are only four possible rooted tree structures and, based on a sample, a choice can be made by performing comparisons between the three bivariate margins of the empirical distribution of the three variables. The s…
There is currently a dearth of appropriate methods to estimate the causal effects of multiple treatments when the outcome is binary. For such settings, we propose the use of nonparametric Bayesian modeling, Bayesian Additive Regression Trees (BART). We conduct an extensive simulation study to compare BART to several existing, propensity score-based methods and to identify its operating characteristics when estimating average treatment effects on the treated. BART consistently demonstrates low bias and mean-squared errors. We illustrate the use of BART through a comparative effectiveness analys…
We present a Bayesian nonparametric model for conditional distribution estimation using Bayesian additive regression trees (BART). The generative model we use is based on rejection sampling from a base model. Typical of BART models, our model is flexible, has a default prior specification, and is computationally convenient. To address the distinguished role of the response in the BART model we propose, we further introduce an approach to targeted smoothing which is possibly of independent interest for BART models. We study the proposed model theoretically and provide sufficient conditions for …
Phylogenetic trees are key data objects in biology, and the method of phylogenetic reconstruction has been highly developed. The space of phylogenetic trees is a nonpositively curved metric space. Recently, statistical methods to analyze the set of trees on this space are being developed utilizing this property. Meanwhile, in Euclidean space, the log-concave maximum likelihood method has emerged as a new nonparametric method for probability density estimation. In this paper, we derive a sufficient condition for the existence and uniqueness of the log-concave maximum likelihood estimator on tre…
We study the problem of maximum likelihood estimation given one data sample ($n=1$) over Brownian Motion Tree Models (BMTMs), a class of Gaussian models on trees. BMTMs are often used as a null model in phylogenetics, where the one-sample regime is common. Specifically, we show that, almost surely, the one-sample BMTM maximum likelihood estimator (MLE) exists, is unique, and corresponds to a fully observed tree. Moreover, we provide a polynomial time algorithm for its exact computation. We also consider the MLE over all possible BMTM tree structures in the one-sample case and show that it exis…
Columbia University（哥伦比亚大学）；Pontifical Catholic University of Rio de Janeiro (PUC-Rio)（里约热内卢天主教大学）；University of California, Irvine（加州大学尔湾分校）
In this paper, we introduce a new machine learning (ML) model for nonlinear regression called the Boosted Smooth Transition Regression Trees (BooST), which is a combination of boosting algorithms with smooth transition regression trees. The main advantage of the BooST model is the estimation of the derivatives (partial effects) of very general nonlinear models. Therefore, the model can provide more interpretation about the mapping between the covariates and the dependent variable than other tree-based models, such as Random Forests. We present several examples with both simulated and real data…
