We investigate the limit behavior of the $L_k$-distance between a decreasing density $f$ and its nonparametric maximum likelihood estimator $\hat{f}_n$ for $k\geq1$. Due to the inconsistency of $\hat{f}_n$ at zero, the case $k=2.5$ turns out to be a kind of transition point. We extend asymptotic normality of the $L_1$-distance to the $L_k$-distance for $1\leq k<2.5$, and obtain the analogous limiting result for a modification of the $L_k$-distance for $k\geq2.5$. Since the $L_1$-distance is the area between $f$ and $\hat{f}_n$, which is also the area between the inverse $g$ of $f$ and the more tractable inverse $U_n$ of $\hat{f}_n$, the problem can be reduced immediately to deriving asymptotic normality of the $L_1$-distance between $U_n$ and $g$. Although we lose this easy correspondence for $k>1$, we show that the $L_k$-distance between $f$ and $\hat{f}_n$ is asymptotically equivalent to the $L_k$-distance between $U_n$ and $g$.
The goal of this paper is to study the bootstrap for the Grenander estimator. The first result is a proof of the inconsistency of the nonparametric bootstrap for the Grenander estimator at a given point. The second result is the development and verification of a bootstrap for the $L_1$ confidence band for the Grenander estimator. As part of this work, kernel estimators are studied as alternatives to the Grenander estimator. We show that when the second derivative of the true density is assumed to be uniformly bounded, there exist kernel estimators with faster convergence rates than the Grenand…
In this paper we consider the stacking of isotonic regression and the method of rearrangement with the empirical estimator to estimate a discrete distribution with an infinite support. The estimators are proved to be strongly consistent with $\sqrt{n}$-rate of convergence. We obtain the asymptotic distributions of the estimators and construct the asymptotically correct conservative global confidence bands. We show that stacked Grenander estimator outperforms the stacked rearrangement estimator. The new estimators behave well even for small sized data sets and provide a trade-off between goodne…
We consider Grenander type estimators for a monotone function $λ:[0,1]\to\mathbb{R}$, obtained as the slope of a concave (convex) estimate of the primitive of $λ$. Our main result is a central limit theorem for the Hellinger loss, which applies to statistical models that satisfy the setup in Durot (2007). This includes estimation of a monotone density, for which the limiting variance of the Hellinger loss turns out to be independent of $λ$.
We consider the Grenander estimator that is the maximum likelihood estimator for non-increasing densities. We prove uniform central limit theorems for certain subclasses of bounded variation functions and for Hölder balls of smoothness s>1/2. We do not assume that the density is differentiable or continuous. The proof can be seen as an adaptation of the method for the parametric maximum likelihood estimator to the nonparametric setting. Since nonparametric maximum likelihood estimators lie on the boundary, the derivative of the likelihood cannot be expected to equal zero as in the parametric c…
We consider the Cox regression model and study the asymptotic global behavior of the Grenander-type estimator for a monotone baseline hazard function. This model is not included in the general setting of Durot (2007). However, we show that a similar central limit theorem holds for $L_p$-error of the Grenander-type estimator. We also propose a test procedure for a Weibull baseline distribution, based on the $L_p$-distance between the Grenander estimator and a parametric estimator of the baseline hazard. Simulation studies are performed to investigate the performance of this test.
Let $f$ be a nonincreasing function defined on $[0,1]$. Under standard regularity conditions, we derive the asymptotic distribution of the supremum norm of the difference between $f$ and its Grenander-type estimator on sub-intervals of $[0,1]$. The rate of convergence is found to be of order $(n/\log n)^{-1/3}$ and the limiting distribution to be Gumbel.
In this paper, we investigate the (in)-consistency of different bootstrap methods for constructing confidence intervals in the class of estimators that converge at rate $n^{1/3}$. The Grenander estimator, the nonparametric maximum likelihood estimator of an unknown nonincreasing density function $f$ on $[0,\infty)$, is a prototypical example. We focus on this example and explore different approaches to constructing bootstrap confidence intervals for $f(t_0)$, where $t_0\in(0,\infty)$ is an interior point. We find that the bootstrap estimate, when generating bootstrap samples from the empirical…
We investigate the asymptotic behavior of the $L_p$-distance between a monotone function on a compact interval and a smooth estimator of this function. Our main result is a central limit theorem for the $L_p$-error of smooth isotonic estimators obtained by smoothing a Grenander-type estimator or isotonizing the ordinary kernel estimator. As a preliminary result we establish a similar result for ordinary kernel estimators. Our results are obtained in a general setting, which includes estimation of a monotone density, regression function and hazard rate. We also perform a simulation study for te…
We consider two isotonic smooth estimators for a monotone baseline hazard in the Cox model, a maximum smooth likelihood estimator and a Grenander-type estimator based on the smoothed Breslow estimator for the cumulative baseline hazard. We show that they are both asymptotically normal at rate $n^{m/(2m+1)}$, where $m\geq 2$ denotes the level of smoothness considered, and we relate their limit behavior to kernel smoothed isotonic estimators studied in Lopuhaä and Musta (2016). It turns out that the Grenander-type estimator is asymptotically equivalent to the kernel smoothed isotonic estimators,…
We investigate nonparametric estimation of a monotone baseline hazard and a decreasing baseline density within the Cox model. Two estimators of a nondecreasing baseline hazard function are proposed. We derive the nonparametric maximum likelihood estimator and consider a Grenander type estimator, defined as the left-hand slope of the greatest convex minorant of the Breslow estimator. We demonstrate that the two estimators are strong consistent and asymptotically equivalent and derive their common limit distribution at a fixed point. Both estimators of a nonincreasing baseline hazard and their a…
In linear models, the generalized least squares (GLS) estimator is applicable when the structure of the error dependence is known. When it is unknown, such structure must be approximated and estimated in a manner that may lead to misspecification. The large-sample analysis of incorrectly-specified GLS (IGLS) estimators requires careful asymptotic manipulations. When performing estimation in the frequency domain, the asymptotic normality of the IGLS estimator, under the so-called Grenander assumptions, has been proved for a broad class of error dependence models. Under the same assumptions, asy…
We consider the smoothed maximum likelihood estimator and the smoothed Grenander-type estimator for a monotone baseline hazard rate $λ_0$ in the Cox model. We analyze their asymptotic behavior and show that they are asymptotically normal at rate $n^{m/(2m+1)}$, when~$λ_0$ is $m\geq 2$ times continuously differentiable, and that both estimators are asymptotically equivalent. Finally, we present numerical results on pointwise confidence intervals that illustrate the comparable behavior of the two methods.
We consider Grenander type estimators for monotone functions $f$ in a very general setting, which includes estimation of monotone regression curves, monotone densities, and monotone failure rates. These estimators are defined as the left-hand slope of the least concave majorant $\hat{F}_n$ of a naive estimator $F_n$ of the integrated curve $F$ corresponding to $f$. We prove that the supremum distance between $\hat{F}_n$ and $F_n$ is of the order $O_p(n^{-1}\log n)^{2/(4-τ)}$, for some $τ\in[0,4)$ that characterizes the tail probabilities of an approximating process for $F_n$. In typical exampl…
In this paper, we propose an estimator of the generalized maximum mean discrepancy between several distributions, constructed by modifying a naive estimator. Asymptotic normality is obtained for this estimator both under equality of these distributions and under the alternative hypothesis.
We propose an estimator of the kernel-based conditional mean dependence measure obtained from an appropriate modification of a naive estimator based on usual empirical estimators. We then get asymptotic normality of this estimator both under conditional mean independence hypothesis and under the alternative hypothesis. A new test for conditional mean independence of random variables valued into Hilbert spaces is then introduced.
We establish asymptotic normality for estimators of the additive regression components under random censorship. To build our estimators, we couple the marginal integration method (Newey (1994)) with an initial Inverse Probability of Censoring Weighted estimator of the multivariate censored regression function introduced by Carbonez et al. (1995) and Kohler et al. (2002). Asymptotic confidence bands are derived from our result.
In this paper, the maximum spacing method is considered for multivariate observations. Nearest neighbour balls are used as a multidimensional analogue to univariate spacings. A class of information-type measures is used to generalize the concept of maximum spacing estimators. Asymptotic normality of these generalized maximum spacing estimators is proved when the assigned model class is correct, that is the true density is a member of the model class.
We derive asymptotic normality of kernel type deconvolution density estimators. In particular we consider deconvolution problems where the known component of the convolution has a symmetric lambda-stable distribution, 0<lambda<= 2. It turns out that the limit behavior changes if the exponent parameter lambda passes the value one, the case of Cauchy deconvolution.
In the present paper we propose a new estimator of entropy based on smooth estimators of quantile density. The consistency and asymptotic distribution of the proposed estimates are obtained. As a consequence, a new test of normality is proposed. A small power comparison is provided. A simulation study for the comparison, in terms of mean squared error, of all estimators under study is performed.
The main purpose of this paper is to estimate the regression function by using a recursive nonparametric kernel approach. We derive the asymptotic normality for a general class of recursive kernel estimate of the regression function, under strong mixing conditions. Our purpose is to extend the work of Roussas and Tran [17] concerning the Devroye-Wagner estimate.
We derive asymptotic normality of kernel type deconvolution estimators of the density, the distribution function at a fixed point, and of the probability of an interval. We consider the so called super smooth case where the characteristic function of the known distribution decreases exponentially. It turns out that the limit behavior of the pointwise estimators of the density and distribution function is relatively straightforward while the asymptotics of the estimator of the probability of an interval depends in a complicated way on the sequence of bandwidths.
This paper establishes a necessary and sufficient condition for the asymptotic normality of the nonparametric estimator of sample coverage proposed by Good [Biometrica 40 (1953) 237--264]. This new necessary and sufficient condition extends the validity of the asymptotic normality beyond the previously proven cases.
We consider in this paper the Grenander estimator of unbounded, in general, nonincreasing densities on the interval [0; 1] without any smoothness assumptions. For fixed number n of i.i.d. random vari- ables X1;X2; : : : ;Xn with values in [0; 1] and the nonincreasing den- sity function f(x), 0 < x < 1, we prove an inequality bounding the weighted integrated square error of the Grenander estimator with probability one by the classical Kolmogorov-Smirnov statistic. Fur- ther, we consider some interesting implications of the latter inequality
To enhance the robustness of the classic least sum of squares (LS) of the residuals estimator, Zuo (2022) introduced the least sum of squares of trimmed (LST) residuals estimator. The LST enjoys many desired properties and serves well as a robust alternative to the LS. Its asymptotic properties, including strong and root-n consistency, have been established whereas the asymptotic normality is left unaddressed. This article solves this remained problem.
We consider the asymptotic normality in $L^2$ of kernel estimators of the long run covariance kernel of stationary functional time series. Our results are established assuming a weakly dependent Bernoulli shift structure for the underlying observations, which contains most stationary functional time series models, under mild conditions. As a corollary, we obtain joint asymptotics for functional principal components computed from empirical long run covariance operators, showing that they have the favorable property of being asymptotically independent.
Strong consistency and asymptotic normality of the Quasi-Maximum Likelihood Estimator (QMLE) are given for a general class of multidimensional causal processes. For particular cases already studied in the literature (for instance univariate or multivariate GARCH, ARCH, ARMA-GARCH processes) the assumptions required for establishing these results are often weaker than existing conditions. The QMLE asymptotic behavior is also given for numerous new examples of univariate or multivariate processes (for instance TARCH or NLARCH processes).
We establish sufficient conditions for the asymptotic normality of kernel density estimators, applied to causal linear random fields. Our conditions on the coefficients of linear random fields are weaker than known results, although our assumption on the bandwidth is not minimal. The proof is based on the $m$-approximation method. As a key step, we prove a central limit theorem for triangular arrays of stationary $m$-dependent random fields with unbounded $m$. We also apply a moment inequality recently established for stationary random fields.
We consider a one dimensional ballistic random walk evolving in a parametric independent and identically distributed random environment. We study the asymptotic properties of the maximum likelihood estimator of the parameter based on a single observation of the path till the time it reaches a distant site. We prove an asymptotic normality result for this consistent estimator as the distant site tends to infinity and establish that it achieves the Cramér-Rao bound. We also explore in a simulation setting the numerical behaviour of asymptotic confidence regions for the parameter value.
The paper concerns the asymptotic distribution of the mixture density estimator, proposed by Oppenheim et al 2006, in the aggregation/disaggregation problem of random parameter AR(1) process. We prove that, under mild conditions on the (semiparametric) form of the mixture density, the estimator is asymptotically normal. The proof is based on the limit theory for the quadratic form in linear random variables developed by Bhansali et al 2007. The moving average representation of the aggregated process is investigated. A small simulation study illustrates the result.
Multivariate processes with long-range dependence properties can be encountered in many fields of application. Two fundamental characteristics in such frameworks are long-range dependence parameters and correlations between component time series. We consider multivariate long-range dependent linear processes, not necessarily Gaussian. We show that the covariances between the wavelet coefficients in this setting are asymptotically Gaussian. We also study the asymptotic distributions of the estimators of the long-range dependence parameter and the long-run covariance by a wavelet-based Whittle p…
The asymptotic normality of the Maximum Likelihood Estimator (MLE) is a cornerstone of statistical theory. In the present paper, we provide sharp explicit upper bounds on Zolotarev-type distances between the exact, unknown distribution of the MLE and its limiting normal distribution. Our approach to this fundamental issue is based on a sound combination of the Delta method, Stein's method, Taylor expansions and conditional expectations, for the classical situations where the MLE can be expressed as a function of a sum of independent and identically distributed terms. This encompasses in partic…
Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstoc…
We establish limit theory for the Grenander estimator of a monotone density near zero. In particular we consider the situation when the true density $f_0$ is unbounded at zero, with different rates of growth to infinity. In the course of our study we develop new switching relations by use of tools from convex analysis. The theory is applied to a problem involving mixtures.
Stochastic simulators such as Monte-Carlo estimators are widely used in science and engineering to study physical systems through their probabilistic representation. Global sensitivity analysis aims to identify the input parameters which have the most important impact on the output. A popular tool to perform global sensitivity analysis is the variance-based method which comes from the Hoeffding-Sobol decomposition. Nevertheless, this method requires an important number of simulations and is often unfeasible under reasonable time constraint. Therefore, an approximation of the input/output relat…
We consider linear processes, not necessarily Gaussian, with long, short or negative memory. The memory parameter is estimated semi-parametrically using wavelets from a sample $X_1,...,X_n$ of the process. We treat both the log-regression wavelet estimator and the wavelet Whittle estimator. We show that these estimators are asymptotically normal as the sample size $n\to\infty$ and we obtain an explicit expression for the limit variance. These results are derived from a general result on the asymptotic normality of the empirical scalogram for linear processes, conveniently centered and normaliz…
We show that spline and wavelet series regression estimators for weakly dependent regressors attain the optimal uniform (i.e. sup-norm) convergence rate $(n/\log n)^{-p/(2p+d)}$ of Stone (1982), where $d$ is the number of regressors and $p$ is the smoothness of the regression function. The optimal rate is achieved even for heavy-tailed martingale difference errors with finite $(2+(d/p))$th absolute moment for $d/p<2$. We also establish the asymptotic normality of t statistics for possibly nonlinear, irregular functionals of the conditional mean function under weak conditions. The results are p…
A multivariable measurement error model $AX \approx B$ is considered. Here $A$ and $B$ are input and output matrices of measurements and $X$ is a rectangular matrix of fixed size to be estimated. The errors in $[A,B]$ are row-wise independent, but within each row the errors may be correlated. Some of the columns are observed without errors and the error covariance matrices may differ from row to row. The total covariance structure of the errors is known up to a scalar factor. The fully weighted total least squares estimator of $X$ is studied. We give conditions for asymptotic normality of the …
We have shown in previous work that statistical inference for cooperative sequential adsorption model can be based on maximum likelihood estimation. In this paper we continue this research and establish asymptotic normality of the maximum likelihood estimator in thermodynamic limit. We also perform and discuss some numerical simulations of the model.
The paper discusses the estimation of a continuous density function of the target random field $X_{\bf{i}}$, $\bf{i}\in \mathbb {Z}^N$ which is contaminated by measurement errors. In particular, the observed random field $Y_{\bf{i}}$, $\bf{i}\in \mathbb {Z}^N$ is such that $Y_{\bf{i}}=X_{\bf{i}}+ε_{\bf{i}}$, where the random error $ε_{\bf{i}}$ is from a known distribution and independent of the target random field. Compared to the existing results, the paper is improved in two directions. First, the random vectors in contrast to univariate random variables are investigated. Second, a random fi…
In this paper, we study the asymptotic normality of the conditional maximum likelihood (ML) estimators for the truncated regression model and the Tobit model. We show that under the general setting assumed in his book, the conjectures made by Hayashi (2000) \footnote{see page 516, and page 520 of Hayashi (2000).} about the asymptotic normality of the conditional ML estimators for both models are true, namely, a sufficient condition is the nonsingularity of $\mathbf{x_tx'_t}$.
Consider $n$ i.i.d. random elements on $C[0,1]$. We show that, under an appropriate strengthening of the domain of attraction condition, natural estimators of the extreme-value index, which is now a continuous function, and the normalizing functions have a Gaussian process as limiting distribution. A key tool is the weak convergence of a weighted tail empirical process, which makes it possible to obtain the results uniformly on $[0,1]$. Detailed examples are also presented.
Statistical analysis of network is an active research area and the literature counts a lot of papers concerned with network models and statistical analysis of networks. However, very few papers deal with missing data in network analysis and we reckon that, in practice, networks are often observed with missing values. In this paper we focus on the Stochastic Block Model with valued edges and consider a MCAR setting by assuming that every dyad (pair of nodes) is sampled identically and independently of the others with probability $ρ> 0$. We prove that maximum likelihood estimators and its variat…
Let $X_1,...,X_n$ be i.i.d. observations, where $X_i=Y_i+σ_n Z_i$ and the $Y$'s and $Z$'s are independent. Assume that the $Y$'s are unobservable and that they have the density $f$ and also that the $Z$'s have a known density $k.$ Furthermore, let $σ_n$ depend on $n$ and let $σ_n\to 0$ as $n\to\infty.$ We consider the deconvolution problem, i.e. the problem of estimation of the density $f$ based on the sample $X_1,...,X_n.$ A popular estimator of $f$ in this setting is the deconvolution kernel density estimator. We derive its asymptotic normality under two different assumptions on the relation…
It is well known that under general regularity conditions the distribution of the maximum likelihood estimator (MLE) is asymptotically normal. Very recently, bounds of the optimal order $O(1/\sqrt n)$ on the closeness of the distribution of the MLE to normality in the so-called bounded Wasserstein distance were obtained, where $n$ is the sample size. However, the corresponding bounds on the Kolmogorov distance were only of the order $O(1/n^{1/4})$. In this note, bounds of the optimal order $O(1/\sqrt n)$ on the closeness of the distribution of the MLE to normality in the Kolmogorov distance ar…
A variety of estimators for the parameters of the Generalized Pareto distribution, the approximating distribution for excesses over a high threshold, have been proposed, always assuming the underlying data to be independent. We recently proved that the likelihood moment estimators are consistent estimators for the parameters of the Generalized Pareto distribution for the case where the underlying data arises from a (stationary) linear process with heavy-tailed innovations. In this paper we derive the bivariate asymptotic normality under some additional assumptions and give an explicit example …
The Latent Block Model (LBM) is a model-based method to cluster simultaneously the $d$ columns and $n$ rows of a data matrix. Parameter estimation in LBM is a difficult and multifaceted problem. Although various estimation strategies have been proposed and are now well understood empirically, theoretical guarantees about their asymptotic behavior is rather sparse and most results are limited to the binary setting. We prove here theoretical guarantees in the valued settings. We show that under some mild conditions on the parameter space, and in an asymptotic regime where $\log(d)/n$ and $\log(n…
The problem of estimating the tail index from truncated data is addressed in Chakrabarty and Samorodnitsky (2009). In that paper, a sample based (and hence random) choice of k is suggested, and it is shown that the choice leads to a consistent estimator of the inverse of the tail index. In this paper, the second order behavior of the Hill estimator with that choice of k is studied, under some additional assumptions. In the untruncated situation, it is well known that asymptotic normality of the Hill estimator follows from the assumption of second order regular variation of the underlying distr…
We prove conditional asymptotic normality of a class of quadratic U-statistics that are dominated by their degenerate second order part and have kernels that change with the number of observations. These statistics arise in the construction of estimators in high-dimensional semi- and non-parametric models, and in the construction of nonparametric confidence sets. This is illustrated by estimation of the integral of a square of a density or regression function, and estimation of the mean response with missing data. We show that estimators are asymptotically normal even in the case that the rate…
We prove the asymptotic normality of the kernel density estimator (introduced by Rosenblatt (1956) and Parzen (1962)) in the context of stationary strongly mixing random fields. Our approach is based on the Lindeberg's method rather than on Bernstein's small-block-large-block technique and coupling arguments widely used in previous works on nonparametric estimation for spatial processes. Our method allows us to consider only minimal conditions on the bandwidth parameter and provides a simple criterion on the (non-uniform) strong mixing coefficients which do not depend on the bandwith.
Linear thresholding models postulate that the conditional distribution of a response variable in terms of covariates differs on the two sides of a (typically unknown) hyperplane in the covariate space. A key goal in such models is to learn about this separating hyperplane. Exact likelihood or least squares methods to estimate the thresholding parameter involve an indicator function which make them difficult to optimize and are, therefore, often tackled by using a surrogate loss that uses a smooth approximation to the indicator. In this paper, we demonstrate that the resulting estimator is asym…
Spectral singularities at non-zero frequencies play an important role in investigating cyclic or seasonal time series. The publication [2] introduced the generalized filtered method-of-moments approach to simultaneously estimate singularity location and long-memory parameters. This paper continues studies of these simultaneous estimators. A wide class of Gegenbauer-type semi-parametric models is considered. Asymptotic normality of several statistics of the cyclic and long-memory parameters is proved. New adjusted estimates are proposed and investigated. The theoretical findings are illustrated…
