We establish that Laplace transforms of the posterior Dirichlet process converge to those of the limiting Brownian bridge process in a neighbourhood about zero, uniformly over Glivenko-Cantelli function classes. For real-valued random variables and functions of bounded variation, we strengthen this result to hold for all real numbers. This last result is proved via an explicit strong approximation coupling inequality.
We provide a general theoretical framework to derive Bernstein-von Mises theorems for matrix functionals. The conditions on functionals and priors are explicit and easy to check. Results are obtained for various functionals including entries of covariance matrix, entries of precision matrix, quadratic forms, log-determinant, eigenvalues in the Bayesian Gaussian covariance/precision matrix estimation setting, as well as for Bayesian linear and quadratic discriminant analysis.
In this paper, we study the asymptotic posterior distribution of linear functionals of the density. In particular, we give general conditions to obtain a semiparametric version of the Bernstein-Von Mises theorem. We then apply this general result to nonparametric priors based on infinite dimensional exponential families. As a byproduct, we also derive adaptive nonparametric rates of concentration of the posterior distributions under these families of priors on the class of Sobolev and Besov spaces.
This paper revisits the prominent Fisher, Wilks, and Bernstein -- von Mises (BvM) results from different viewpoints. Particular issues to address are: nonasymptotic framework with just one finite sample, possible model misspecification, and a large parameter dimension. In particular, in the case of an i.i.d. sample, the mentioned results can be stated for any smooth parametric family provided that the dimension \(p \) of the parameter space satisfies the condition "\(p^{2}/n \) is small" for the Fisher expansion, while the Wilks and the BvM results require "\(p^{3}/n \) is small".
This note considers a finite dimensional statistical model for the Calderón problem with piecewise constant conductivities. In this setting it is shown that injectivity of the forward map and its linearisation suffice to prove the invertibility of the information operator, resulting in a Bernstein--von-Mises theorem and optimality guarantees for estimation by Bayesian posterior means.
We prove a Bernstein-von Mises theorem for a general class of high dimensional nonlinear Bayesian inverse problems in the vanishing noise limit. We propose a sufficient condition on the growth rate of the number of unknown parameters under which the posterior distribution is asymptotically normal. This growth condition is expressed explicitly in terms of the model dimension, the degree of ill-posedness of the inverse problem and the noise parameter. The theoretical results are applied to a Bayesian estimation of the medium parameter in an elliptic problem.
We investigate Bernstein-von Mises theorems for adaptive nonparametric Bayesian procedures in the canonical Gaussian white noise model. We consider both a Hilbert space and multiscale setting with applications in $L^2$ and $L^\infty$ respectively. This provides a theoretical justification for plug-in procedures, for example the use of certain credible sets for sufficiently smooth linear functionals. We use this general approach to construct optimal frequentist confidence sets based on the posterior distribution. We also provide simulations to numerically illustrate our approach and obtain a vi…
We continue the investigation of Bernstein-von Mises theorems for nonparametric Bayes procedures from [Ann. Statist. 41 (2013) 1999-2028]. We introduce multiscale spaces on which nonparametric priors and posteriors are naturally defined, and prove Bernstein-von Mises theorems for a variety of priors in the setting of Gaussian nonparametric regression and in the i.i.d. sampling model. From these results we deduce several applications where posterior-based inference coincides with efficient frequentist procedures, including Donsker- and Kolmogorov-Smirnov theorems for the random posterior cumula…
In the recent Bayesian nonparametric literature, many examples have been reported in which Bayesian estimators and posterior distributions do not achieve the optimal convergence rate, indicating that the Bernstein-von Mises theorem does not hold. In this article, we give a positive result in this direction by showing that the Bernstein-von Mises theorem holds in survival models for a large class of prior processes neutral to the right. We also show that, for an arbitrarily given convergence rate n^{-α} with 0<α\leq 1/2, a prior process neutral to the right can be chosen so that its posterior d…
We demonstrate that a prior influence on the posterior distribution of covariance matrix vanishes as sample size grows. The assumptions on a prior are explicit and mild. The results are valid for a finite sample and admit the dimension $p$ growing with the sample size $n$. We exploit the described fact to derive the finite sample Bernstein-von Mises theorem for functionals of covariance matrix (e.g. eigenvalues) and to find the posterior distribution of the Frobenius distance between spectral projector and empirical spectral projector. This can be useful for constructing sharp confidence sets …
This paper brings a contribution to the Bayesian theory of nonparametric and semiparametric estimation. We are interested in the asymptotic normality of the posterior distribution in Gaussian linear regression models when the number of regressors increases with the sample size. Two kinds of Bernstein-von Mises Theorems are obtained in this framework: nonparametric theorems for the parameter itself, and semiparametric theorems for functionals of the parameter. We apply them to the Gaussian sequence model and to the regression of functions in Sobolev and $C^α$ classes, in which we get the minima…
In this paper, we first study a new sensitivity index that is based on higher moments and generalizes the so-called Sobol one. Further, following an idea of Borgonovo ([3]), we define and study a new sensitivity index based on the Cram{é}r von Mises distance. This new index appears to be more general than the Sobol one as it takes into account, not only the variance, but the whole distribution of the random variable. Furthermore, we study the statistical properties of a Monte Carlo estimate of this new index.
Statistical methods for functional data are of interest for many applications. In this paper, we prove a central limit theorem for random variables taking their values in a Hilbert space. The random variables are assumed to be weakly dependent in the sense of near epoch dependence, where the underlying process fulfills some mixing conditions. As parametric inference in an infinite dimensional space is difficult, we show that the nonoverlapping block bootstrap is consistent. Furthermore, we show how these results can be used for degenerate von Mises-statistics.
We study large sample properties of Bayesian analysis of the proportional hazard model with neutral to the right process priors on the baseline hazard function. We show that the posterior distribution of the baseline cumulative hazard function and regression coefficients centered at the maximum likelihood estimator is jointly asymptotically equivalent to the sampling distribution of the maximum likelihood estimator.
A Bernstein-von Mises theorem is derived for general semiparametric functionals. The result is applied to a variety of semiparametric problems in i.i.d. and non-i.i.d. situations. In particular, new tools are developed to handle semiparametric bias, in particular for nonlinear functionals and in cases where regularity is possibly low. Examples include the squared $L^2$-norm in Gaussian white noise, nonlinear functionals in density estimation, as well as functionals in autoregressive models. For density estimation, a systematic study of BvM results for two important classes of priors is provide…
Bernstein-von Mises theorems for nonparametric Bayes priors in the Gaussian white noise model are proved. It is demonstrated how such results justify Bayes methods as efficient frequentist inference procedures in a variety of concrete nonparametric problems. Particularly Bayesian credible sets are constructed that have asymptotically exact $1-α$ frequentist coverage level and whose $L^2$-diameter shrinks at the minimax rate of convergence (within logarithmic factors) over Hölder balls. Other applications include general classes of linear and nonlinear functionals and credible bands for auto-co…
In the first part of the paper we study stochastic integrals of a nonrandom function with respect to a nonorthogonal Hilbert noise defined on a semiring of subsets of an arbitrary nonempty set. In the second part we apply this construction to study limit behavior of canonical (i.e., degenerate) Von Mises statistics based on weakly dependent stationary observations.
In a smooth semi-parametric model, the marginal posterior distribution for a finite dimensional parameter of interest is expected to be asymptotically equivalent to the sampling distribution of any efficient point-estimator. The assertion leads to asymptotic equivalence of credible and confidence sets for the parameter of interest and is known as the semi-parametric Bernstein-von Mises theorem. In recent years, it has received much attention and has been applied in many examples. We consider models in which errors with symmetric densities play a role; more specifically, it is shown that the ma…
This chapter presents some novel information theoretic results for the analysis of stationary time series in the frequency domain. In particular, the spectral distribution that corresponds to the most uncertain or unpredictable time series with some values of the autocovariance function fixed, is the generalized von Mises spectral distribution. It is thus a maximum entropy spectral distribution and the corresponding stationary time series is called the generalized von Mises time series. The generalized von Mises distribution is used in directional statistics for modelling planar directions tha…
The major goal of this paper is to study the second order frequentist properties of the marginal posterior distribution of the parametric component in semiparametric Bayesian models, in particular, a second order semiparametric Bernstein-von Mises (BvM) Theorem. Our first contribution is to discover an interesting interference phenomenon between Bayesian estimation and frequentist inferential accuracy: more accurate Bayesian estimation on the nuisance function leads to higher frequentist inferential accuracy on the parametric component. As the second contribution, we propose a new class of dep…
In application areas like bioinformatics multivariate distributions on angles are encountered which show significant clustering. One approach to statistical modelling of such situations is to use mixtures of unimodal distributions. In the literature (Mardia et al., 2011), the multivariate von Mises distribution, also known as the multivariate sine distribution, has been suggested for components of such models, but work in the area has been hampered by the fact that no good criteria for the von Mises distribution to be unimodal were available. In this article we study the question about when a …
We study the asymptotic behaviour of the posterior distribution in a broad class of statistical models where the "true" solution occurs on the boundary of the parameter space. We show that in this case Bayesian inference is consistent, and that the posterior distribution has not only Gaussian components as in the case of regular models (the Bernstein-von Mises theorem) but also has Gamma distribution components whose form depends on the behaviour of the prior distribution near the boundary and have a faster rate of convergence. We also demonstrate a remarkable property of Bayesian inference, t…
The classical parametric and semiparametric Bernstein -- von Mises (BvM) results are reconsidered in a non-classical setup allowing finite samples and model misspecification. In the case of a finite dimensional nuisance parameter we obtain an upper bound on the error of Gaussian approximation of the posterior distribution for the target parameter which is explicit in the dimension of the nuisance and target parameters. This helps to identify the so called \emph{critical dimension} $ p $ of the full parameter for which the BvM result is applicable. In the important i.i.d. case, we show that the…
We point out an error in the proof of the main result of the paper of Tanabe et al. (2007) concerning a parameter estimation for von Mises--Fisher distributions, we correct the proof of the main result and we present a short alternative proof.
We consider the statistical inverse problem of recovering an unknown function $f$ from a linear measurement corrupted by additive Gaussian white noise. We employ a nonparametric Bayesian approach with standard Gaussian priors, for which the posterior-based reconstruction of $f$ corresponds to a Tikhonov regulariser $\bar f$ with a reproducing kernel Hilbert space norm penalty. We prove a semiparametric Bernstein-von Mises theorem for a large collection of linear functionals of $f$, implying that semiparametric posterior estimation and uncertainty quantification are valid and optimal from a fre…
In this work, nonparametric log-rank-type statistical tests are introduced in order to verify homogeneity of purely discrete variables subject to arbitrary right-censoring for infinitely many categories. In particular, the Cramér-von Mises test statistics for discrete models under censoring is established. In order to introduce the test, we develop the weighted log-rank statistics in a general multivariate discrete setup which complements previous fundamental results of Gill (1980) and Andersen et al. (1982). Due to the presence of persistent jumps over the unbounded set of categories, the asy…
Bayesian tests on the symmetry of the generalized von Mises model for planar directions (Gatto and Jammalamadaka, 2007) are introduced. The generalized von Mises distribution is a flexible model that can be axially symmetric or asymmetric, unimodal or bimodal. A characterization of axial symmetry is provided and taken as null hypothesis for one of the proposed Bayesian tests. The Bayesian tests are obtained by the technique of probability perturbation. The prior probability measure is perturbed so to give a positive prior probability to the null hypothesis, which would be null otherwise. This …
Let $F_N$ and $F$ be the empirical and limiting spectral distributions of an $N\times N$ Wigner matrix. The Cramér-von Mises (CvM) statistic is a classical goodness-of-fit statistic that characterizes the distance between $F_N$ and $F$ in $\ell^2$-norm. In this paper, we consider a mesoscopic approximation of the CvM statistic for Wigner matrices, and derive its limiting distribution. In the appendix, we also give the limiting distribution of the CvM statistic (without approximation) for the toy model CUE.
In a smooth semiparametric model, the marginal posterior distribution of the finite dimensional parameter of interest is expected to be asymptotically equivalent to the sampling distribution of frequentist's efficient estimators. This is the assertion of the so-called Bernstein-von Mises theorem, and recently, it has been proved in many interesting semiparametric models. In this thesis, we consider the semiparametric Bernstein-von Mises theorem in some models which have symmetric errors. The simplest example of these models is the symmetric location model that has 1-dimensional location parame…
Few methods in Bayesian non-parametric statistics/ machine learning have received as much attention as Bayesian Additive Regression Trees (BART). While BART is now routinely performed for prediction tasks, its theoretical properties began to be understood only very recently. In this work, we continue the theoretical investigation of BART initiated by Rockova and van der Pas (2017). In particular, we study the Bernstein-von Mises (BvM) phenomenon (i.e. asymptotic normality) for smooth linear functionals of the regression surface within the framework of non-parametric regression with fixed covar…
We study nonparametric Bayesian statistical inference for the parameters governing a pure jump process of the form $$Y_t = \sum_{k=1}^{N(t)} Z_k,~~~ t \ge 0,$$ where $N(t)$ is a standard Poisson process of intensity $λ$, and $Z_k$ are drawn i.i.d.~from jump measure $μ$. A high-dimensional wavelet series prior for the Lévy measure $ν= λμ$ is devised and the posterior distribution arises from observing discrete samples $Y_Δ, Y_{2Δ}, \dots, Y_{nΔ}$ at fixed observation distance $Δ$, giving rise to a nonlinear inverse inference problem. We derive contraction rates in uniform norm for the posterior…
In a smooth semiparametric estimation problem, the marginal posterior for the parameter of interest is expected to be asymptotically normal and satisfy frequentist criteria of optimality if the model is endowed with a suitable prior. It is shown that, under certain straightforward and interpretable conditions, the assertion of Le Cam's acclaimed, but strictly parametric, Bernstein-von Mises theorem [Univ. California Publ. Statist. 1 (1953) 277-329] holds in the semiparametric situation as well. As a consequence, Bayesian point-estimators achieve efficiency, for example, in the sense of Hájek's…
Probability distributions in Stiefel manifold such as the von-Mises Fisher and Bingham distributions find diverse applications in signal processing and other applied sciences. Use of these statistical models in practice is complicated by the difficulties in numerical evaluation of their normalization constants. In this letter, we derive asymptotical approximations to the normalization constants via recent results in random matrix theory. The derived approximations take simple forms and are reasonably accurate in regimes of practical interest. As an application, we show that the proposed analyt…
This paper studies circular correlations for the bivariate von Mises sine and cosine distributions. These are two simple and appealing models for bivariate angular data with five parameters each that have interpretations comparable to those in the ordinary bivariate normal model. However, the variability and association of the angle pairs cannot be easily deduced from the model parameters unlike the bivariate normal. Thus to compute such summary measures, tools from circular statistics are needed. We derive analytic expressions and study the properties of the Jammalamadaka-Sarma and Fisher-Lee…
The prominent Bernstein -- von Mises (BvM) result claims that the posterior distribution after centering by the efficient estimator and standardizing by the square root of the total Fisher information is nearly standard normal. In particular, the prior completely washes out from the asymptotic posterior distribution. This fact is fundamental and justifies the Bayes approach from the frequentist viewpoint. In the nonparametric setup the situation changes dramatically and the impact of prior becomes essential even for the contraction of the posterior; see [vdV2008], [Bo2011], [CaNi2013,CaNi2014]…
We introduce some new classes of unimodal rotational invariant directional distributions, which generalize von Mises-Fisher distribution. We propose three types of distributions, one of which represents axial data. For each new type we provide formulae and short computational study of parameter estimators by the method of moments and the method of maximum likelihood. The main goal of the paper is to develop the goodness of fit test to detect that sample entries follow one of the introduced generalized von Mises--Fisher distribution based on the maximum entropy principle. We use $k$th nearest n…
