We give a sufficient condition for admissibility of generalized Bayes estimators of the location vector of spherically symmetric distribution under squared error loss. Compared to the known results for the multivariate normal case, our sufficient condition is very tight and is close to being a necessary condition. In particular we establish the admissibility of generalized Bayes estimators with respect to the harmonic prior and priors with slightly heavier tail than the harmonic prior. We use the theory of regularly varying functions to construct a sequence of smooth proper priors approaching an improper prior fast enough for establishing the admissibility. We also discuss conditions of minimaxity of the generalized Bayes estimator with respect to the harmonic prior.
In this article, we consider two forms of shrinkage estimators of the mean $θ$ of a multivariate normal distribution $X\sim N_{p}\left(θ, σ^{2}I_{p}\right)$ where $σ^{2}$ is unknown. We take the prior law $θ\sim N_{p}\left(\upsilon, τ^{2}I_{p}\right)$ and we constuct a Modified Bayes estimator $δ_{B}^{\ast}$ and an Empirical Modified Bayes estimator $δ_{EB}^{\ast}$. We are interested in studying the minimaxity and the limits of risks ratios of these estimators, to the maximum likelihood estimator $X$, when $n$ and $p$ tend to infinity.
Shrinkage prior are becoming more and more popular in Bayesian modeling for high dimensional sparse problems due to its computational efficiency. Recent works show that a polynomially decaying prior leads to satisfactory posterior asymptotics under regression models. In the literature, statisticians have investigated how the global shrinkage parameter, i.e., the scale parameter, in a heavy tail prior affects the posterior contraction. In this work, we explore how the shape of the prior, or more specifically, the polynomial order of the prior tail affects the posterior. We discover that, under …
We study admissibility of a subclass of generalized Bayes estimators of a multivariate normal vector when the variance is unknown, under scaled quadratic loss. Minimaxity is also established for certain of these estimators.
This paper reviews minimax best equivariant estimation in these invariant estimation problems: a location parameter, a scale parameter and a (Wishart) covariance matrix. We briefly review development of the best equivariant estimator as a generalized Bayes estimator relative to right invariant Haar measure in each case. Then we prove minimaxity of the best equivariant procedure by giving a least favorable prior sequence based on non-truncated Gaussian distributions. The results in this paper are all known, but we bring a fresh and somewhat unified approach by using, in contrast to most proofs …
In this article, we investigate certain asymptotic optimality properties of a very broad class of one-group continuous shrinkage priors for simultaneous estimation and testing of a sparse normal mean vector. Asymptotic optimality of Bayes estimates and posterior concentration properties corresponding to the general class of one-group priors under consideration are studied where the data is assumed to be generated according to a multivariate normal distribution with a fixed unknown mean vector. Under the assumption that the number of non-zero means is known, we show that Bayes estimators arisin…
In the estimation of the mean matrix in a multivariate normal distribution, the generalized Bayes estimators with closed forms are provided, and the sufficient conditions for their minimaxity are derived relative to both matrix and scalar quadratic loss functions. The generalized Bayes estimators of the covariance matrix are also given with closed forms, and the dominance properties are discussed for the Stein loss function.
Brown's 1971 paper "Admissible estimators, recurrent diffusions and insoluble boundary value problems" is a landmark in the admissibility literature. It nearly completely settles the issue of admissibility/inadmissibility for estimating the mean of a multivariate normal distribution with identity covariance under sum of squared error loss. We revisit this wonderful tour de force on its 50th anniversary and present an alternative and more direct proof of the result for generalized Bayes estimators corresponding to priors which are a subclass of scale mixtures of spherical normals.
In this paper, we consider the estimation of a mean vector of a multivariate normal population where the mean vector is suspected to be nearly equal to mean vectors of $k-1$ other populations. As an alternative to the preliminary test estimator based on the test statistic for testing hypothesis of equal means, we derive empirical and hierarchical Bayes estimators which shrink the sample mean vector toward a pooled mean estimator given under the hypothesis. The minimaxity of those Bayesian estimators are shown, and their performances are investigated by simulation.
We consider Bayesian shrinkage predictions for the Normal regression problem under the frequentist Kullback-Leibler risk function. Firstly, we consider the multivariate Normal model with an unknown mean and a known covariance. While the unknown mean is fixed, the covariance of future samples can be different from training samples. We show that the Bayesian predictive distribution based on the uniform prior is dominated by that based on a class of priors if the prior distributions for the covariance and future covariance matrices are rotation invariant. Then, we consider a class of priors for t…
We evaluate priors by the second order asymptotic behavior of the corresponding estimators.Under certain regularity conditions, the risk differences between efficient estimators of parameters taking values in a domain D, an open connected subset of R^d, are asymptotically expressed as elliptic differential forms depending on the asymptotic covariance matrix V. Each efficient estimator has the same asymptotic risk as a 'local Bayes' estimate corresponding to a prior density p. The asymptotic decision theory of the estimators identifies the smooth prior densities as admissible or inadmissible, a…
We consider a Bayesian approach to model selection in Gaussian linear regression, where the number of predictors might be much larger than the number of observations. From a frequentist view, the proposed procedure results in the penalized least squares estimation with a complexity penalty associated with a prior on the model size. We investigate the optimality properties of the resulting estimator. We establish the oracle inequality and specify conditions on the prior that imply its asymptotic minimaxity within a wide range of sparse and dense settings for "nearly-orthogonal" and "multicollin…
Many real-life applications involve estimation of curves that exhibit complicated shapes including jumps or varying-frequency oscillations. Practical methods have been devised that can adapt to a locally varying complexity of an unknown function (e.g. variable-knot splines, sparse wavelet reconstructions, kernel methods or trees/forests). However, the overwhelming majority of existing asymptotic minimaxity theory is predicated on homogeneous smoothness assumptions. Focusing on locally Holderian functions, we provide new locally adaptive posterior concentration rate results under the supremum l…
We consider admissibility of generalized Bayes estimators of the mean of a multivariate normal distribution when the scale is unknown under quadratic loss. The priors considered put the improper invariant prior on the scale while the prior on the mean has a hierarchical normal structure conditional on the scale. This conditional hierarchical prior is essentially that of Maruyama and Strawderman (2021, Biometrika) (MS21) which is indexed by a hyperparameter $a$. In that paper $a$ is chosen so this conditional prior is proper which corresponds to $a>-1$. This paper extends MS21 by considering im…
Consider two independent exponential populations having different unknown location parameters and a common unknown scale parameter. Call the population associated with the larger location parameter as the "best" population and the population associated with the smaller location parameter as the "worst" population. For the goal of selecting the best (worst) population a natural selection rule, that has many optimum properties, is the one which selects the population corresponding to the larger (smaller) minimal sufficient statistic. In this article, we consider the problem of estimating the loc…
Recent decades have seen an interest in prediction problems for which Bayesian methodology has been used ubiquitously. Sampling from or approximating the posterior predictive distribution in a Bayesian model allows one to make inferential statements about potentially observable random quantities given observed data. The purpose of this note is to use statistical decision theory as a basis to justify the use of a posterior predictive distribution for making a point prediction.
We consider evaluating improper priors in a formal Bayes setting according to the consequences of their use. Let $Φ$ be a class of functions on the parameter space and consider estimating elements of $Φ$ under quadratic loss. If the formal Bayes estimator of every function in $Φ$ is admissible, then the prior is strongly admissible with respect to $Φ$. Eaton's method for establishing strong admissibility is based on studying the stability properties of a particular Markov chain associated with the inferential setting. In previous work, this was handled differently depending upon whether $ϕ\in …
We study predictive density estimation under Kullback-Leibler loss in $\ell_0$-sparse Gaussian sequence models. We propose proper Bayes predictive density estimates and establish asymptotic minimaxity in sparse models. A surprise is the existence of a phase transition in the future-to-past variance ratio $r$. For $r < r_0 = (\surd 5 - 1)/4$, the natural discrete prior ceases to be asymptotically optimal. Instead, for subcritical $r$, a `bi-grid' prior with a central region of reduced grid spacing recovers asymptotic minimaxity. This phenomenon seems to have no analog in the otherwise parallel …
We consider estimation of a multivariate normal mean vector under sum of squared error loss. We propose a new class of smooth estimators parameterized by αdominating the James-Stein estimator. The estimator for α=1 corresponds to the generalized Bayes estimator with respect to the harmonic prior. When αgoes to infinity, the estimator converges to the James-Stein positive-part estimator. Thus the class of our estimators is a bridge between the admissible estimator (α=1) and the inadmissible estimator (α=\infty). Although the estimators have quasi-admissibility which is a weaker optimality than …
In this paper we consider the nonparametric functional estimation of the drift of Gaussian processes using Paley-Wiener and Karhunen-Loève expansions. We construct efficient estimators for the drift of such processes, and prove their minimaxity using Bayes estimators. We also construct superefficient estimators of Stein type for such drifts using the Malliavin integration by parts formula and stochastic analysis on Gaussian space, in which superharmonic functionals of the process paths play a particular role. Our results are illustrated by numerical simulations and extend the construction of J…
This article discusses estimation of a multivariate normal mean based on heteroscedastic observations. Under heteroscedasticity, estimators shrinking more on the coordinates with larger variances, seem desirable. Although they are not necessarily minimax in the ordinary sense, we show that such James-Stein type estimators can be ensemble minimax, minimax with respect to the ensemble risk, related to empirical Bayes perspective of Efron and Morris.
For finite parameter spaces under finite loss, every Bayes procedure derived from a prior with full support is admissible, and every admissible procedure is Bayes. This relationship already breaks down once we move to finite-dimensional Euclidean parameter spaces. Compactness and strong regularity conditions suffice to repair the relationship, but without these conditions, admissible procedures need not be Bayes. Under strong regularity conditions, admissible procedures can be shown to be the limits of Bayes procedures. Under even stricter conditions, they are generalized Bayes, i.e., they min…
This paper is devoted to the multivariate estimation of a vector of Poisson means. A novel loss function that penalises bad estimates of each of the parameters and the sum (or equivalently the mean) of the parameters is introduced. Under this loss function, a class of minimax estimators that uniformly dominate the maximum likelihood estimator is derived. Crucially, these methods have the property that for estimating a given component parameter, the full data vector is utilised. Estimators in this class can be fine-tuned to limit shrinkage away from the maximum likelihood estimator, thereby avo…
In a remarkable series of papers beginning in 1956, Charles Stein set the stage for the future development of minimax shrinkage estimators of a multivariate normal mean under quadratic loss. More recently, parallel developments have seen the emergence of minimax shrinkage estimators of multivariate normal predictive densities under Kullback--Leibler risk. We here describe these parallels emphasizing the focus on Bayes procedures and the derivation of the superharmonic conditions for minimaxity as well as further developments of new minimax shrinkage predictive density estimators including mult…
We give an exact characterization of admissibility in statistical decision problems in terms of Bayes optimality in a so-called nonstandard extension of the original decision problem, as introduced by Duanmu and Roy. Unlike the consideration of improper priors or other generalized notions of Bayes optimalitiy, the nonstandard extension is distinguished, in part, by having priors that can assign "infinitesimal" mass in a sense that can be made rigorous using results from nonstandard analysis. With these additional priors, we find that, informally speaking, a decision procedure $δ_0$ is admissib…
In statistical exercises where there are several candidate models, the traditional approach is to select one model using some data driven criterion and use that model for estimation, testing and other purposes, ignoring the variability of the model selection process. We discuss some problems associated with this approach. An alternative scheme is to use a model-averaged estimator, that is, a weighted average of estimators obtained under different models, as an estimator of a parameter. We show that the risk associated with a Bayesian model-averaged estimator is bounded as a function of the sam…
Consider a parametric statistical model $P(\mathrm{d}x|θ)$ and an improper prior distribution $ν(\mathrm{d}θ)$ that together yield a (proper) formal posterior distribution $Q(\mathrm{d}θ|x)$. The prior is called strongly admissible if the generalized Bayes estimator of every bounded function of $θ$ is admissible under squared error loss. Eaton [Ann. Statist. 20 (1992) 1147--1179] has shown that a sufficient condition for strong admissibility of $ν$ is the local recurrence of the Markov chain whose transition function is $R(θ,\mathrm{d}η)=\int Q(\mathrm{d}η|x)P(\mathrm {d}x|θ)$. Applications of…
Consider the problem of estimating a multivariate normal mean with a known variance matrix, which is not necessarily proportional to the identity matrix. The coordinates are shrunk directly in proportion to their variances in Efron and Morris' (J. Amer. Statist. Assoc. 68 (1973) 117-130) empirical Bayes approach, whereas inversely in proportion to their variances in Berger's (Ann. Statist. 4 (1976) 223-226) minimax estimators. We propose a new minimax estimator, by approximately minimizing the Bayes risk with a normal prior among a class of minimax estimators where the shrinkage direction is o…
