We establish limit theory for the Grenander estimator of a monotone density near zero. In particular we consider the situation when the true density $f_0$ is unbounded at zero, with different rates of growth to infinity. In the course of our study we develop new switching relations by use of tools from convex analysis. The theory is applied to a problem involving mixtures.
The goal of this paper is to study the bootstrap for the Grenander estimator. The first result is a proof of the inconsistency of the nonparametric bootstrap for the Grenander estimator at a given point. The second result is the development and verification of a bootstrap for the $L_1$ confidence band for the Grenander estimator. As part of this work, kernel estimators are studied as alternatives to the Grenander estimator. We show that when the second derivative of the true density is assumed to be uniformly bounded, there exist kernel estimators with faster convergence rates than the Grenand…
We consider in this paper the Grenander estimator of unbounded, in general, nonincreasing densities on the interval [0; 1] without any smoothness assumptions. For fixed number n of i.i.d. random vari- ables X1;X2; : : : ;Xn with values in [0; 1] and the nonincreasing den- sity function f(x), 0 < x < 1, we prove an inequality bounding the weighted integrated square error of the Grenander estimator with probability one by the classical Kolmogorov-Smirnov statistic. Fur- ther, we consider some interesting implications of the latter inequality
In this paper we consider the stacking of isotonic regression and the method of rearrangement with the empirical estimator to estimate a discrete distribution with an infinite support. The estimators are proved to be strongly consistent with $\sqrt{n}$-rate of convergence. We obtain the asymptotic distributions of the estimators and construct the asymptotically correct conservative global confidence bands. We show that stacked Grenander estimator outperforms the stacked rearrangement estimator. The new estimators behave well even for small sized data sets and provide a trade-off between goodne…
We consider Grenander type estimators for a monotone function $λ:[0,1]\to\mathbb{R}$, obtained as the slope of a concave (convex) estimate of the primitive of $λ$. Our main result is a central limit theorem for the Hellinger loss, which applies to statistical models that satisfy the setup in Durot (2007). This includes estimation of a monotone density, for which the limiting variance of the Hellinger loss turns out to be independent of $λ$.
We consider the Grenander estimator that is the maximum likelihood estimator for non-increasing densities. We prove uniform central limit theorems for certain subclasses of bounded variation functions and for Hölder balls of smoothness s>1/2. We do not assume that the density is differentiable or continuous. The proof can be seen as an adaptation of the method for the parametric maximum likelihood estimator to the nonparametric setting. Since nonparametric maximum likelihood estimators lie on the boundary, the derivative of the likelihood cannot be expected to equal zero as in the parametric c…
We consider the Cox regression model and study the asymptotic global behavior of the Grenander-type estimator for a monotone baseline hazard function. This model is not included in the general setting of Durot (2007). However, we show that a similar central limit theorem holds for $L_p$-error of the Grenander-type estimator. We also propose a test procedure for a Weibull baseline distribution, based on the $L_p$-distance between the Grenander estimator and a parametric estimator of the baseline hazard. Simulation studies are performed to investigate the performance of this test.
Let $f$ be a nonincreasing function defined on $[0,1]$. Under standard regularity conditions, we derive the asymptotic distribution of the supremum norm of the difference between $f$ and its Grenander-type estimator on sub-intervals of $[0,1]$. The rate of convergence is found to be of order $(n/\log n)^{-1/3}$ and the limiting distribution to be Gumbel.
In this note we prove the following law of the iterated logarithm for the Grenander estimator of a monotone decreasing density: If $f(t_0) > 0$, $f'(t_0) < 0$, and $f'$ is continuous in a neighborhood of $t_0$, then \begin{eqnarray*} \limsup_{n\rightarrow \infty} \left ( \frac{n}{2\log \log n} \right )^{1/3} ( \widehat{f}_n (t_0 ) - f(t_0) ) = \left| f(t_0) f'(t_0)/2 \right|^{1/3} 2M \end{eqnarray*} almost surely where $ M \equiv \sup_{g \in {\cal G}} T_g = (3/4)^{1/3}$ and $ T_g \equiv \mbox{argmax}_u \{ g(u) - u^2 \} $; here ${\cal G}$ is the two-sided Strassen limit set on $R$. The proof re…
In this paper, we investigate the (in)-consistency of different bootstrap methods for constructing confidence intervals in the class of estimators that converge at rate $n^{1/3}$. The Grenander estimator, the nonparametric maximum likelihood estimator of an unknown nonincreasing density function $f$ on $[0,\infty)$, is a prototypical example. We focus on this example and explore different approaches to constructing bootstrap confidence intervals for $f(t_0)$, where $t_0\in(0,\infty)$ is an interior point. We find that the bootstrap estimate, when generating bootstrap samples from the empirical…
We investigate the asymptotic behavior of the $L_p$-distance between a monotone function on a compact interval and a smooth estimator of this function. Our main result is a central limit theorem for the $L_p$-error of smooth isotonic estimators obtained by smoothing a Grenander-type estimator or isotonizing the ordinary kernel estimator. As a preliminary result we establish a similar result for ordinary kernel estimators. Our results are obtained in a general setting, which includes estimation of a monotone density, regression function and hazard rate. We also perform a simulation study for te…
We investigate the limit behavior of the $L_k$-distance between a decreasing density $f$ and its nonparametric maximum likelihood estimator $\hat{f}_n$ for $k\geq1$. Due to the inconsistency of $\hat{f}_n$ at zero, the case $k=2.5$ turns out to be a kind of transition point. We extend asymptotic normality of the $L_1$-distance to the $L_k$-distance for $1\leq k<2.5$, and obtain the analogous limiting result for a modification of the $L_k$-distance for $k\geq2.5$. Since the $L_1$-distance is the area between $f$ and $\hat{f}_n$, which is also the area between the inverse $g$ of $f$ and the more…
We consider two isotonic smooth estimators for a monotone baseline hazard in the Cox model, a maximum smooth likelihood estimator and a Grenander-type estimator based on the smoothed Breslow estimator for the cumulative baseline hazard. We show that they are both asymptotically normal at rate $n^{m/(2m+1)}$, where $m\geq 2$ denotes the level of smoothness considered, and we relate their limit behavior to kernel smoothed isotonic estimators studied in Lopuhaä and Musta (2016). It turns out that the Grenander-type estimator is asymptotically equivalent to the kernel smoothed isotonic estimators,…
Under the assumption that the true density is decreasing, it is well known that the Grenander estimator converges at rate $n^{1/3}$ if the true density is curved [Sankhyā Ser. A 31 (1969) 23-36] and at rate $n^{1/2}$ if the density is flat [Ann. Probab. 11 (1983) 328-345; Canad. J. Statist. 27 (1999) 557-566]. In the case that the true density is misspecified, the results of Patilea [Ann. Statist. 29 (2001) 94-123] tell us that the global convergence rate is of order $n^{1/3}$ in Hellinger distance. Here, we show that the local convergence rate is $n^{1/2}$ at a point where the density is miss…
We investigate nonparametric estimation of a monotone baseline hazard and a decreasing baseline density within the Cox model. Two estimators of a nondecreasing baseline hazard function are proposed. We derive the nonparametric maximum likelihood estimator and consider a Grenander type estimator, defined as the left-hand slope of the greatest convex minorant of the Breslow estimator. We demonstrate that the two estimators are strong consistent and asymptotically equivalent and derive their common limit distribution at a fixed point. Both estimators of a nonincreasing baseline hazard and their a…
We propose a new estimator of a discrete monotone probability mass function with known flat regions. We analyse its asymptotic properties and compare its performance to the Grenander estimator and to the monotone rearrangement estimator.
We consider the smoothed maximum likelihood estimator and the smoothed Grenander-type estimator for a monotone baseline hazard rate $λ_0$ in the Cox model. We analyze their asymptotic behavior and show that they are asymptotically normal at rate $n^{m/(2m+1)}$, when~$λ_0$ is $m\geq 2$ times continuously differentiable, and that both estimators are asymptotically equivalent. Finally, we present numerical results on pointwise confidence intervals that illustrate the comparable behavior of the two methods.
We consider Grenander type estimators for monotone functions $f$ in a very general setting, which includes estimation of monotone regression curves, monotone densities, and monotone failure rates. These estimators are defined as the left-hand slope of the least concave majorant $\hat{F}_n$ of a naive estimator $F_n$ of the integrated curve $F$ corresponding to $f$. We prove that the supremum distance between $\hat{F}_n$ and $F_n$ is of the order $O_p(n^{-1}\log n)^{2/(4-τ)}$, for some $τ\in[0,4)$ that characterizes the tail probabilities of an approximating process for $F_n$. In typical exampl…
The problem of nonparametric inference on a monotone function has been extensively studied in many particular cases. Estimators considered have often been of so-called Grenander type, being representable as the left derivative of the greatest convex minorant or least concave majorant of an estimator of a primitive function. In this paper, we provide general conditions for consistency and pointwise convergence in distribution of a class of generalized Grenander-type estimators of a monotone function. This broad class allows the minorization or majoratization operation to be performed on a data-…
In this paper we will consider the estimation of a monotone regression (or density) function in a fixed point by the least squares (Grenander) estimator. We will show that this estimator is fully adaptive, in the sense that the attained rate is given by a functional relation using the underlying function $f_0$, and not by some smoothness parameter, and that this rate is optimal when considering the class of all monotone functions, in the sense that there exists a sequence of alternative monotone functions $f_1$, such that no other estimator can attain a better rate for both $f_0$ and $f_1$. We…
We aim at estimating a function $λ:[0,1]\to \mathbb {R}$, subject to the constraint that it is decreasing (or increasing). We provide a unified approach for studying the $\mathbb {L}_p$-loss of an estimator defined as the slope of a concave (or convex) approximation of an estimator of a primitive of $λ$, based on $n$ observations. Our main task is to prove that the $\mathbb {L}_p$-loss is asymptotically Gaussian with explicit (though unknown) asymptotic mean and variance. We also prove that the local $\mathbb {L}_p$-risk at a fixed point and the global $\mathbb {L}_p$-risk are of order $n^{-p/…
We consider kernel smoothed Grenander-type estimators for a monotone hazard rate and a monotone density in the presence of randomly right censored data. We show that they converge at rate $n^{2/5}$ and that the limit distribution at a fixed point is Gaussian with explicitly given mean and variance. It is well-known that standard kernel smoothing leads to inconsistency problems at the boundary points. It turns out that, also by using a boundary correction, we can only establish uniform consistency on intervals that stay away from the end point of the support (though we can go arbitrarily close …
The estimation of the Lévy density, the infinite-dimensional parameter controlling the jump dynamics of a Lévy process, is considered here under a discrete-sampling scheme. In this setting, the jumps are latent variables, the statistical properties of which can be assessed when the frequency and time horizon of observations increase to infinity at suitable rates. Nonparametric estimators for the Lévy density based on Grenander's method of sieves was proposed in Figueroa-López [IMS Lecture Notes 57 (2009) 117--146]. In this paper, central limit theorems for these sieve estimators, both pointwis…
We study a high-dimensional regression setting under the assumption of known covariate distribution. We aim at estimating the amount of explained variation in the response by the best linear function of the covariates (the signal level). In our setting, neither sparsity of the coefficient vector, nor normality of the covariates or linearity of the conditional expectation are assumed. We present an unbiased and consistent estimator and then improve it by using a zero-estimator approach, where a zero-estimator is a statistic whose expected value is zero. More generally, we present an algorithm b…
In this paper, we extend the work of Pimentel et al. (2015) and propose an adjusted estimator of Kendall's $τ$ for bivariate zero-inflated count data. We provide achievable lower and upper bounds of our proposed estimator and show its relationship with current literature. In addition, we also suggest an estimator of the achievable bounds, thereby helping practitioners interpret the results while working with real data. The performance of the proposed estimator for Kendall's $τ$ is unbiased with smaller mean squared errors compared to the unadjusted estimator of Pimentel et al. (2015). Our resu…
We study a linear high-dimensional regression model in a semi-supervised setting, where for many observations only the vector of covariates $X$ is given with no response $Y$. We do not make any sparsity assumptions on the vector of coefficients, and aim at estimating $\mathrm{Var}(Y|X)$. We propose an estimator, which is unbiased, consistent, and asymptotically normal. This estimator can be improved by adding zero-estimators arising from the unlabelled data. Adding zero-estimators does not affect the bias and potentially can reduce variance. In order to achieve optimal improvement, many zero-e…
In this paper, we consider an inference problem for the first order autoregressive process with non-zero mean driven by a long memory stationary Gaussian process. Suppose that the covariance function of the noise can be expressed as $|k|^{2H-2}$ times a positive constant when $k$ tends to infinity, and the fractional Gaussian noise and the fractional ARIMA model are special examples that satisfy this assumption. We propose moment estimators and prove the strong consistency, the asymptotic normality and joint asymptotic normality.
This paper concerns with the group zero-norm regularized least squares estimator which, in terms of the variational characterization of the zero-norm, can be obtained from a mathematical program with equilibrium constraints (MPEC). By developing the global exact penalty for the MPEC, this estimator is shown to arise from an exact penalization problem that not only has a favorable bilinear structure but also implies a recipe to deliver equivalent DC estimators such as the SCAD and MCP estimators. We propose a multi-stage convex relaxation approach (GEP-MSCRA) for computing this estimator, and u…
Many statistical practices involve choosing between a full model and reduced models where some coefficients are reduced to zero. Data were used to select a model with estimated coefficients. Is it possible to do so and still come up with an estimator always better than the traditional estimator based on the full model? The James-Stein estimator is such an estimator, having a property called minimaxity. However, the estimator considers only one reduced model, namely the origin. Hence it reduces no coefficient estimator to zero or every coefficient estimator to zero. In many applications includi…
We study the asymptotic properties of bridge estimators in sparse, high-dimensional, linear regression models when the number of covariates may increase to infinity with the sample size. We are particularly interested in the use of bridge estimators to distinguish between covariates whose coefficients are zero and covariates whose coefficients are nonzero. We show that under appropriate conditions, bridge estimators correctly select covariates with nonzero coefficients with probability converging to one and that the estimators of nonzero coefficients have the same asymptotic distribution that …
This paper proposes a max-test for testing (possibly infinitely) many zero parameter restrictions in an extremum estimation framework. The test statistic is formed by estimating key parameters one at a time based on many empirical loss functions that map from a low dimension parameter space, and choosing the largest in absolute value from these individually estimated parameters. The parsimoniously parametrized loss identify whether the original parameter of interest is or is not zero. Estimating fixed low dimension sub-parameters ensures greater estimator accuracy, does not require a sparsity …
Asymptotic properties of three estimators of probability density function of sample maximum $f_{(m)}:=mfF^{m-1}$ are derived, where $m$ is a function of sample size $n$. One of the estimators is the parametrically fitted by the approximating generalized extreme value density function. However, the parametric fitting is misspecified in finite $m$ cases. The misspecification comes from mainly the following two: the difference $m$ and the selected block size $k$, and the poor approximation $f_{(m)}$ to the generalized extreme value density which depends on the magnitude of $m$ and the extreme ind…
The problem of estimating the ratio of the means of a two-component Poisson mixture model is considered, when each component is subject to zero-inflation, i.e., excess zero counts. The. resulting {\it zero-inflated Poisson mixture (ZIPM) model} can be treated as a three-component Poisson mixture model with one degenerate component. The EM algorithm is applied to obtain frequentist estimators and their standard errors, the latter determined via an explicit expression for the observed information matrix. Bayes and empirical Bayes estimators also are obtained by means of conjugate priors and thei…
Under mild conditions, it is shown the strong consistency of the Bayes estimator of the density. Moreover, the Bayes risk (for some common loss functions) of the Bayes estimator of the density (i.e. the posterior predictive density) reaches zero when the sample size goes to $\infty$. In passing, a similar result is obtained for the estimation of the sampling distribution.
In this paper, we propose a new regularization technique called "functional SCAD". We then combine this technique with the smoothing spline method to develop a smooth and locally sparse (i.e., zero on some sub-regions) estimator for the coefficient function in functional linear regression. The functional SCAD has a nice shrinkage property that enables our estimating procedure to identify the null subregions of the coefficient function without over shrinking the non-zero values of the coefficient function. Additionally, the smoothness of our estimated coefficient function is regularized by a ro…
In this work we study, as the temperature goes to zero, the oscillation of Metropolis-Hasting's algorithm around the Basis Pursuit De-noising solutions. We derive new criteria for choosing the proposal distribution and the temperature in Metropolis-Hasting's algorithm. Finally we apply these results to compare Metropolis-Hasting's and simulated annealing algorithms.
In this paper we establish the uniform in bandwidth consistency for the transformation kernel estimator of copulas introduced in [Omelka et al.(2009)]. To this end, we first prove a uniform in bandwidth law of the iterated logarithm for the maximal deviation of this estimator from its expectation. We then show that, as n goes to infinity, the bias of the estimator converges to zero uniformly in the bandwidth h, varying over a suitable interval. A practical method of selecting the optimal bandwidth is also presented. Finally, we make conclusive simulation experiments showing the performance of …
It is shown that the Hall, Hu and Marron [Hall, P., Hu, T., and Marron J.S. (1995), Improved Variable Window Kernel Estimates of Probability Densities, {\it Annals of Statistics}, 23, 1--10] modification of Abramson's [Abramson, I. (1982), On Bandwidth Variation in Kernel Estimates - A Square-root Law, {\it Annals of Statistics}, 10, 1217--1223] variable bandwidth kernel density estimator satisfies the optimal asymptotic properties for estimating densities with four uniformly continuous derivatives, uniformly on bounded sets where the preliminary estimator of the density is bounded away from z…
This work studies online zero-order optimization of convex and Lipschitz functions. We present a novel gradient estimator based on two function evaluations and randomization on the $\ell_1$-sphere. Considering different geometries of feasible sets and Lipschitz assumptions we analyse online dual averaging algorithm with our estimator in place of the usual gradient. We consider two types of assumptions on the noise of the zero-order oracle: canceling noise and adversarial noise. We provide an anytime and completely data-driven algorithm, which is adaptive to all parameters of the problem. In th…
In this paper we propose a generalization of a class of Gaussian Semiparametric Estimators (GSE) of the fractional differencing parameter for long-range dependent multivariate time series. We generalize a known GSE-type estimator by introducing some modifications at the objective function level regarding the process' spectral density matrix estimator. We study large sample properties of the estimator without assuming Gaussianity as well as hypothesis testing. The class of models considered here satisfies simple conditions on the spectral density function, restricted to a small neighborhood of …
We propose new nonparametric estimators of the integrated volatility of an Itô semimartingale observed at discrete times on a fixed time interval with mesh of the observation grid shrinking to zero. The proposed estimators achieve the optimal rate and variance of estimating integrated volatility even in the presence of infinite variation jumps when the latter are stochastic integrals with respect to locally "stable" Lévy processes, that is, processes whose Lévy measure around zero behaves like that of a stable process. On a first step, we estimate locally volatility from the empirical characte…
In a finite mixture of location-scale distributions maximum likelihood estimator does not exist because of the unboundedness of the likelihood function when the scale parameter of some mixture component approaches zero. In order to study the strong consistency of maximum likelihood estimator, we consider the case that the scale parameters of the component distributions are restricted from below by $c_n$, where $c_n$ is a sequence of positive real numbers which tend to zero as the sample size $n$ increases. We prove that under mild regularity conditions maximum likelihood estimator is strongly …
The present paper studies density deconvolution in the presence of small Berkson errors, in particular, when the variances of the errors tend to zero as the sample size grows. It is known that when the Berkson errors are present, in some cases, the unknown density estimator can be obtain by simple averaging without using kernels. However, this may not be the case when Berkson errors are asymptotically small. By treating the former case as a kernel estimator with the zero bandwidth, we obtain the optimal expressions for the bandwidth. We show that the density of Berkson errors acts as a regular…
In this article, we revisit the problem of estimating the unknown zero-symmetric distribution in a two-component location mixture model, considered in previous works, now under the assumption that the zero-symmetric distribution has a log-concave density. When consistent estimators for the shift locations and mixing probability are used, we show that the nonparametric log-concave Maximum Likelihood estimator (MLE) of both the mixed density and that of the unknown zero-symmetric component are consistent in the Hellinger distance. In case the estimators for the shift locations and mixing probabi…
Recently, Ghysels, Hill, and Motegi (2020) proposed a test for examining whether a large number of coefficients in linear regression models is zero. The test is called the max test. The test statistic is calculated by first running multiple ordinary least squares (OLS) regressions, each including only one of key regressors, whose coefficients are supposed to be zero under the null, and then taking the maximum value of the squared OLS coefficient estimates of those key regressors. They called these regressions parsimonious regressions. This paper answers a question raised in their Remark 2.4; w…
The Gini index is a popular inequality measure with many applications in social and economic studies. This paper studies semiparametric inference on the Gini indices of two semicontinuous populations. We characterize the distribution of each semicontinuous population by a mixture of a discrete point mass at zero and a continuous skewed positive component. A semiparametric density ratio model is then employed to link the positive components of the two distributions. We propose the maximum empirical likelihood estimators of the two Gini indices and their difference, and further investigate the a…
In this paper, we investigate the consistency and asymptotic efficiency of an estimator of the drift matrix, $F$, of Ornstein-Uhlenbeck processes that are not necessarily stable. We consider all the cases. (1) The eigenvalues of $F$ are in the right half space (i.e., eigenvalues with positive real parts). In this case the process grows exponentially fast. (2) The eigenvalues of $F$ are on the left half space (i.e., the eigenvalues with negative or zero real parts). The process where all eigenvalues of $F$ have negative real parts is called a stable process and has a unique invariant (i.e., sta…