In this paper, we develop a multistage approach for estimating the mean of a bounded variable. We first focus on the multistage estimation of a binomial parameter and then generalize the estimation methods to the case of general bounded random variables. A fundamental connection between a binomial parameter and the mean of a bounded variable is established. Our multistage estimation methods rigorously guarantee prescribed levels of precision and confidence.
Louisiana State University（路易斯安那州立大学）；Southern University and A&M College（南方大学及农工学院）
In this paper, we have established a unified framework of multistage parameter estimation. We demonstrate that a wide variety of statistical problems such as fixed-sample-size interval estimation, point estimation with error control, bounded-width confidence intervals, interval estimation following hypothesis testing, construction of confidence sequences, can be cast into the general framework of constructing sequential random intervals with prescribed coverage probabilities. We have developed exact methods for the construction of such sequential random intervals in the context of multistage s…
Multi-stage (designed) procedures, obtained by splitting the sampling budget suitably across stages, and designing the sampling at a particular stage based on information about the parameter obtained from previous stages, are often advantageous from the perspective of precise inference. We develop a generic framework for M-estimation in a multistage setting and apply empirical process techniques to develop limit theorems that describe the large sample behavior of the resulting M-estimates. Applications to change-point estimation, inverse isotonic regression, classification and mode estimation …
The $\ell_1$-penalized method, or the Lasso, has emerged as an important tool for the analysis of large data sets. Many important results have been obtained for the Lasso in linear regression which have led to a deeper understanding of high-dimensional statistical problems. In this article, we consider a class of weighted $\ell_1$-penalized estimators for convex loss functions of a general form, including the generalized linear models. We study the estimation, prediction, selection and sparsity properties of the weighted $\ell_1$-penalized estimator in sparse, high-dimensional settings where t…
In this paper, we have developed new multistage tests which guarantee prescribed level of power and are more efficient than previous tests in terms of average sampling number and the number of sampling operations. Without truncation, the maximum sampling numbers of our testing plans are absolutely bounded. Based on geometrical arguments, we have derived extremely tight bounds for the operating characteristic function. To reduce the computational complexity for the relevant integrals, we propose adaptive scanning algorithms which are not only useful for present hypothesis testing problem but al…
In this article, we derive an explicit formula for computing confidence interval for the mean of a bounded random variable. Moreover, we have developed multistage point estimation methods for estimating the mean value with prescribed precision and confidence level based on the proposed confidence interval.
We consider the problem of locating a jump discontinuity (change-point) in a smooth parametric regression model with a bounded covariate. It is assumed that one can sample the covariate at different values and measure the corresponding responses. Budget constraints dictate that a total of $n$ such measurements can be obtained. A multistage adaptive procedure is proposed, where at each stage an estimate of the change point is obtained and new points are sampled from its appropriately chosen neighborhood. It is shown that such procedures accelerate the rate of convergence of the least squares es…
In this paper, we develop a general approach for probabilistic estimation and optimization. An explicit formula and a computational approach are established for controlling the reliability of probabilistic estimation based on a mixed criterion of absolute and relative errors. By employing the Chernoff-Hoeffding bound and the concept of sampling, the minimization of a probabilistic function is transformed into an optimization problem amenable for gradient descendent algorithms.
For the basic maximum likelihood estimating function of the two parameters Weibull distribution, a simple proof on its global monotonicity is given to ensure the existence and uniqueness of its solution. The boundary of the function's first-order derivative is defined based on its scale-free property. With a bounded derivative, the possible range of the root of this function can be determined. A novel root-finding algorithm employing these established results is proposed accordingly, its convergence is proved analytically as well. Compared with other typical algorithms for this problem, the ef…
We study the estimation, in Lp-norm, of density functions defined on [0,1]^d. We construct a new family of kernel density estimators that do not suffer from the so-called boundary bias problem and we propose a data-driven procedure based on the Goldenshluger and Lepski approach that jointly selects a kernel and a bandwidth. We derive two estimators that satisfy oracle-type inequalities. They are also proved to be adaptive over a scale of anisotropic or isotropic Sobolev-Slobodetskii classes (which are particular cases of Besov or Sobolev classical classes). The main interest of the isotropic p…
We consider a parameter estimation problem for one dimensional stochastic heat equations, when data is sampled discretely in time or spatial component. We prove that, the real valued parameter next to the Laplacian (the drift), and the constant parameter in front of the noise (the volatility) can be consistently estimated under somewhat surprisingly minimal information. Namely, it is enough to observe the solution at a fixed time and on a discrete spatial grid, or at a fixed space point and at discrete time instances of a finite interval, assuming that the mesh-size goes to zero. The proposed …
In this paper, we study two problems: (1) estimation of a $d$-dimensional log-concave distribution and (2) bounded multivariate convex regression with random design with an underlying log-concave density or a compactly supported distribution with a continuous density. First, we show that for all $d \ge 4$ the maximum likelihood estimators of both problems achieve an optimal risk of $Θ_d(n^{-2/(d+1)})$ (up to a logarithmic factor) in terms of squared Hellinger distance and $L_2$ squared distance, respectively. Previously, the optimality of both these estimators was known only for $d\le 3$. We a…
In this paper, we introduce the BMT distribution as an unimodal alternative to continuous univariate distributions supported on a bounded interval. The ideas behind the mathematical formulation of this new distribution come from computer aid geometric design, specifically from Bezier curves. First, we review general properties of a distribution given by parametric equations and extend the definition of a Bezier distribution. Then, after proposing the BMT cumulative distribution function, we derive its probability density function and a closed-form expression for quantile function, median, inte…
For normal canonical models with $X \sim N_p(θ, σ^{2} I_{p}), \;\; S^{2} \sim σ^{2}χ^{2}_{k}, \;{independent}$, we consider the problem of estimating $θ$ under scale invariant squared error loss $\frac{\|d-θ\|^{2}}{σ^{2}}$, when it is known that the signal-to-noise ratio $\frac{\|θ\|}σ$ is bounded above by $m$. Risk analysis is achieved by making use of a conditional risk decomposition and we obtain in particular sufficient conditions for an estimator to dominate either the unbiased estimator $δ_{UB}(X)=X$, or the maximum likelihood estimator $δ_{\hbox{mle}}(X,S^2)$, or both of these benchmark…
We propose an estimator for the mean of random variables in separable real Banach spaces using the empirical characteristic function. Assuming that the covariance operator of the random variable is bounded in a precise sense, we show that the proposed estimator achieves the optimal sub-Gaussian rate, except for a faster decaying mean-dependent term. Under a mild condition, an iterative refinement procedure can essentially eliminate the mean-dependent term and provide a shift-equivariant estimate. Our results particularly suggests that a certain Gaussian width that appears in the best known rat…
We consider the problem of estimating a multivariate function $f_0$ of bounded variation (BV), from noisy observations $y_i = f_0(x_i) + z_i$ made at random design points $x_i \in \mathbb{R}^d$, $i=1,\ldots,n$. We study an estimator that forms the Voronoi diagram of the design points, and then solves an optimization problem that regularizes according to a certain discrete notion of total variation (TV): the sum of weighted absolute differences of parameters $θ_i,θ_j$ (which estimate the function values $f_0(x_i),f_0(x_j)$) at all neighboring cells $i,j$ in the Voronoi diagram. This is seen to …
Columbia University, Department of Statistics（哥伦比亚大学统计系）；ENSAE ParisTech, Department of Statistics（巴黎高等经济商业学院统计系）
We derive concentration inequalities for differentially private median and mean estimators building on the "Propose, Test, Release" (PTR) mechanism introduced by Dwork and Lei (2009). We introduce a new general version of the PTR mechanism that allows us to derive high probability error bounds for differentially private estimators. Our algorithms provide the first statistical guarantees for differentially private estimation of the median and mean without any boundedness assumptions on the data, and without assuming that the target population parameter lies in some known bounded interval. Our p…
