Let $f$ be a nonincreasing function defined on $[0,1]$. Under standard regularity conditions, we derive the asymptotic distribution of the supremum norm of the difference between $f$ and its Grenander-type estimator on sub-intervals of $[0,1]$. The rate of convergence is found to be of order $(n/\log n)^{-1/3}$ and the limiting distribution to be Gumbel.
In this paper we consider the stacking of isotonic regression and the method of rearrangement with the empirical estimator to estimate a discrete distribution with an infinite support. The estimators are proved to be strongly consistent with $\sqrt{n}$-rate of convergence. We obtain the asymptotic distributions of the estimators and construct the asymptotically correct conservative global confidence bands. We show that stacked Grenander estimator outperforms the stacked rearrangement estimator. The new estimators behave well even for small sized data sets and provide a trade-off between goodne…
We consider Grenander type estimators for a monotone function $λ:[0,1]\to\mathbb{R}$, obtained as the slope of a concave (convex) estimate of the primitive of $λ$. Our main result is a central limit theorem for the Hellinger loss, which applies to statistical models that satisfy the setup in Durot (2007). This includes estimation of a monotone density, for which the limiting variance of the Hellinger loss turns out to be independent of $λ$.
We consider the Grenander estimator that is the maximum likelihood estimator for non-increasing densities. We prove uniform central limit theorems for certain subclasses of bounded variation functions and for Hölder balls of smoothness s>1/2. We do not assume that the density is differentiable or continuous. The proof can be seen as an adaptation of the method for the parametric maximum likelihood estimator to the nonparametric setting. Since nonparametric maximum likelihood estimators lie on the boundary, the derivative of the likelihood cannot be expected to equal zero as in the parametric c…
We consider the Cox regression model and study the asymptotic global behavior of the Grenander-type estimator for a monotone baseline hazard function. This model is not included in the general setting of Durot (2007). However, we show that a similar central limit theorem holds for $L_p$-error of the Grenander-type estimator. We also propose a test procedure for a Weibull baseline distribution, based on the $L_p$-distance between the Grenander estimator and a parametric estimator of the baseline hazard. Simulation studies are performed to investigate the performance of this test.
We investigate the asymptotic behavior of the $L_p$-distance between a monotone function on a compact interval and a smooth estimator of this function. Our main result is a central limit theorem for the $L_p$-error of smooth isotonic estimators obtained by smoothing a Grenander-type estimator or isotonizing the ordinary kernel estimator. As a preliminary result we establish a similar result for ordinary kernel estimators. Our results are obtained in a general setting, which includes estimation of a monotone density, regression function and hazard rate. We also perform a simulation study for te…
We consider two isotonic smooth estimators for a monotone baseline hazard in the Cox model, a maximum smooth likelihood estimator and a Grenander-type estimator based on the smoothed Breslow estimator for the cumulative baseline hazard. We show that they are both asymptotically normal at rate $n^{m/(2m+1)}$, where $m\geq 2$ denotes the level of smoothness considered, and we relate their limit behavior to kernel smoothed isotonic estimators studied in Lopuhaä and Musta (2016). It turns out that the Grenander-type estimator is asymptotically equivalent to the kernel smoothed isotonic estimators,…
We consider the smoothed maximum likelihood estimator and the smoothed Grenander-type estimator for a monotone baseline hazard rate $λ_0$ in the Cox model. We analyze their asymptotic behavior and show that they are asymptotically normal at rate $n^{m/(2m+1)}$, when~$λ_0$ is $m\geq 2$ times continuously differentiable, and that both estimators are asymptotically equivalent. Finally, we present numerical results on pointwise confidence intervals that illustrate the comparable behavior of the two methods.
We consider Grenander type estimators for monotone functions $f$ in a very general setting, which includes estimation of monotone regression curves, monotone densities, and monotone failure rates. These estimators are defined as the left-hand slope of the least concave majorant $\hat{F}_n$ of a naive estimator $F_n$ of the integrated curve $F$ corresponding to $f$. We prove that the supremum distance between $\hat{F}_n$ and $F_n$ is of the order $O_p(n^{-1}\log n)^{2/(4-τ)}$, for some $τ\in[0,4)$ that characterizes the tail probabilities of an approximating process for $F_n$. In typical exampl…
We consider in this paper the Grenander estimator of unbounded, in general, nonincreasing densities on the interval [0; 1] without any smoothness assumptions. For fixed number n of i.i.d. random vari- ables X1;X2; : : : ;Xn with values in [0; 1] and the nonincreasing den- sity function f(x), 0 < x < 1, we prove an inequality bounding the weighted integrated square error of the Grenander estimator with probability one by the classical Kolmogorov-Smirnov statistic. Fur- ther, we consider some interesting implications of the latter inequality
In linear models, the generalized least squares (GLS) estimator is applicable when the structure of the error dependence is known. When it is unknown, such structure must be approximated and estimated in a manner that may lead to misspecification. The large-sample analysis of incorrectly-specified GLS (IGLS) estimators requires careful asymptotic manipulations. When performing estimation in the frequency domain, the asymptotic normality of the IGLS estimator, under the so-called Grenander assumptions, has been proved for a broad class of error dependence models. Under the same assumptions, asy…
In this note we prove the following law of the iterated logarithm for the Grenander estimator of a monotone decreasing density: If $f(t_0) > 0$, $f'(t_0) < 0$, and $f'$ is continuous in a neighborhood of $t_0$, then \begin{eqnarray*} \limsup_{n\rightarrow \infty} \left ( \frac{n}{2\log \log n} \right )^{1/3} ( \widehat{f}_n (t_0 ) - f(t_0) ) = \left| f(t_0) f'(t_0)/2 \right|^{1/3} 2M \end{eqnarray*} almost surely where $ M \equiv \sup_{g \in {\cal G}} T_g = (3/4)^{1/3}$ and $ T_g \equiv \mbox{argmax}_u \{ g(u) - u^2 \} $; here ${\cal G}$ is the two-sided Strassen limit set on $R$. The proof re…
In this paper, we consider function-indexed normalized weighted integrated periodograms for equidistantly sampled multivariate continuous-time state space models which are multivariate continuous-time ARMA processes. Thereby, the sampling distance is fixed and the driving Lévy process has at least a finite fourth moment. Under different assumptions on the function space and the moments of the driving Lévy process we derive a central limit theorem for the function-indexed normalized weighted integrated periodogram. Either the assumption on the function space or the assumption on the existence o…
We prove the consistency and asymptotic normality of the Laplacian Quasi-Maximum Likelihood Estimator (QMLE) for a general class of causal time series including ARMA, AR($\infty$), GARCH, ARCH($\infty$), ARMA-GARCH, APARCH, ARMA-APARCH,..., processes. We notably exhibit the advantages (moment order and robustness) of this estimator compared to the classical Gaussian QMLE. Numerical simulations confirms the accuracy of this estimator.
The Chebyshev or $\ell_{\infty}$ estimator is an unconventional alternative to the ordinary least squares in solving linear regressions. It is defined as the minimizer of the $\ell_{\infty}$ objective function \begin{align*} \hat{\boldsymbolβ} := \arg\min_{\boldsymbolβ} \|\boldsymbol{Y} - \mathbf{X}\boldsymbolβ\|_{\infty}. \end{align*} The asymptotic distribution of the Chebyshev estimator under fixed number of covariates was recently studied (Knight, 2020), yet finite sample guarantees and generalizations to high-dimensional settings remain open. In this paper, we develop non-asymptotic upper…
We propose a new estimator of a discrete monotone probability mass function with known flat regions. We analyse its asymptotic properties and compare its performance to the Grenander estimator and to the monotone rearrangement estimator.
Estimating the effective dimension reduction (EDR) space, related to the semiparametric regression model introduced by Li \cite{sir}, is based on the estimation of the covariance matrix $Λ$ of the conditional expectation of the vector of predictors given the response. An estimator $\widehatΛ_n$ of $Λ$ based on kernel method was introduced by Zhu and Fang \cite{Asymptotics} who then derived, under some conditions, the asymptotic distribution of $\sqrt{n}\left(\widehatΛ_n-Λ\right)$, as $n\rightarrow +\infty$. In this paper, we obtain, under specified conditions, the almost sure convergence of $\…
This paper aims at providing statistical guarantees for a kernel based estimation of time varying parameters driving the dynamic of local stationary processes. We extend the results of Dahlhaus et al. (2018) considering the local stationary version of the infinite memory processes of Doukhan and Wintenberger (2008). The estimators are computed as localized M-estimators of any contrast satisfying appropriate contraction conditions. We prove the uniform consistency and pointwise asymptotic normality of such kernel based estimators. We apply our result to usual contrasts such as least-square, lea…
The subject of this paper is the problem of estimating service time distribution of the $M/G/\infty$ queue from incomplete data on the queue. The goal is to estimate $G$ from observations of the queue--length process at the points of the regular grid on a fixed time interval. We propose an estimator and analyze its accuracy over a family of target service time distributions. The original $M/G/\infty$ problem is closely related to the problem of estimating derivatives of the covariance function of a stationary Gaussian process. We consider the latter problem and derive lower bounds on the minim…
Consider nonparametric function estimation under $L^p$-loss. The minimax rate for estimation of the regression function over a Hölder ball with smoothness index $β$ is $n^{-β/(2β+1)}$ if $1\leq p<\infty$ and $(n/\log n)^{-β/(2β+1)}$ if $p=\infty.$ There are many known procedures that either attain this rate for $p=\infty$ but are suboptimal by a $\log n$ factor in the case $p<\infty$ or the other way around. In this article, we construct an estimator that simultaneously achieves the optimal rates under $L^p$-risk for all $1\leq p\leq \infty$ without prior knowledge of $β.$ In contrast to class…
In this paper, we study the local constant and the local linear estimators of the conditional density function with right-censored data which exhibit some type of dependence. It is assumed that the observations form a stationary $α-$mixing sequence. The asymptotic normality of the two estimators is established, which combined with the condition that $\lim\limits_{n\rightarrow\infty}nh_nb_n=\infty$ implies the consistency of the two estimators and can be employed to construct confidence intervals for the conditional density function. The result on the local linear estimator of the conditional d…
We consider a reflected Ornstein-Uhlenbeck process $X$ driven by a fractional Brownian motion with Hurst parameter $H\in (0, \frac12) \cup (\frac12, 1)$. Our goal is to estimate an unknown drift parameter $α\in (-\infty,\infty)$ on the basis of continuous observation of the state process. We establish Girsanov theorem for the process $X$, derive the standard maximum likelihood estimator of the drift parameter $α$, and prove its strong consistency and asymptotic normality. As an improved estimator, we obtain the explicit formulas for the sequential maximum likelihood estimator and its mean squa…
The aim of this paper is to provide a new estimator of parameters for LARCH$(\infty)$ processes, and thus also for LARCH$(p)$ or GLARCH$(p,q)$ processes. This estimator results from minimising a contrast leading to a least squares estimator for the absolute values of the process. Strong consistency and asymptotic normality are shown, and convergence occurs at the rate $\sqrt n$ as well in short or long memory cases. Numerical experiments confirm the theoretical results and show that this new estimator significantly outperforms the smoothed quasi-maximum likelihood estimators or weighted least …
We consider the non-parametric Poisson regression problem where the integer valued response $Y$ is the realization of a Poisson random variable with parameter $λ(X)$. The aim is to estimate the functional parameter $λ$ from independent or weakly dependent observations $(X_1,Y_1),\ldots,(X_n,Y_n)$ in a random design framework. First we determine upper risk bounds for projection estimators on finite dimensional subspaces under mild conditions. In the case of Sobolev ellipsoids the obtained rates of convergence turn out to be optimal. The main part of the paper is devoted to the construction of a…
Recently attention has been drawn to practical problems with the use of unbounded Pareto distributions, for instance when there are natural upper bounds that truncate the probability tail. Aban, Meerschaert and Panorska (2006) derived the maximum likelihood estimator for the Pareto tail index of a truncated Pareto distribution with a right truncation point $T$. The Hill (1975) estimator is then obtained by letting $T \to \infty$. The problem of extreme value estimation under right truncation was also introduced in Nuyts (2010) who proposed a similar estimator for the tail index and considered …
Online monotone density estimation and log-optimal calibration
Department of Statistics and Data Science, Carnegie Mellon University, USA（统计与数据科学系，卡内基梅隆大学，美国）；Department of Statistics and Actuarial Science, University of Waterloo, Canada（统计与精算科学系，滑铁卢大学，加拿大）
This paper considers the asymptotic behavior in $β$-Hölder spaces, and under $L^p$ losses, of a Dirichlet kernel density estimator proposed by Aitchison and Lauder (1985) for the analysis of compositional data. In recent work, Ouimet and Tolosana-Delgado (2022) established the uniform strong consistency and asymptotic normality of this estimator. As a complement, it is shown here that the Aitchison-Lauder estimator can achieve the minimax rate asymptotically for a suitable choice of bandwidth whenever $(p,β) \in [1, 3) \times (0, 2]$ or $(p, β) \in \mathcal{A}_d$, where $\mathcal{A}_d$ is a sp…
Estimation of the service time distribution in the discrete-time $GI/G/\infty$-queue based solely on information on the arrival and departure processes is considered. The focus is put on the estimation approach via the so called "sequence of differences". Existing results for this approach are substantially extended by proving a functional central limit theorem for the resultant estimator. Here, the underlying function space is taken to be the space of sequences converging to zero. The moving block bootstrap technique is considered for the estimation of the resultant covariance kernel and is s…
As a starting point we prove a functional central limit theorem for estimators of the invariant measure of a geometrically ergodic Harris-recurrent Markov chain in a multi-scale space. This allows to construct confidence bands for the invariant density with optimal (up to undersmoothing) $L^{\infty}$-diameter by using wavelet projection estimators. In addition our setting applies to the drift estimation of diffusions observed discretely with fixed observation distance. We prove a functional central limit theorem for estimators of the drift function and finally construct adaptive confidence ban…
Nonparametric density estimators are studied for $d$-dimensional, strongly spatial mixing data which is defined on a general $N$-dimensional lattice structure. We consider linear and nonlinear hard thresholded wavelet estimators which are derived from a $d$-dimensional multiresolution analysis. We give sufficient criteria for the consistency of these estimators and derive rates of convergence in $L^{p'}$ for $p'\in [1,\infty)$. For this reason, we study density functions which are elements of a $d$-dimensional Besov space $B^s_{p,q}(\mathbb{R}^d)$. We also verify the analytic correctness of ou…
We consider an epidemic change-point detection in a large class of causal time series models, including among other processes, AR($\infty$), ARCH($\infty$), TARCH($\infty$), ARMA-GARCH. A test statistic based on the Gaussian quasi-maximum likelihood estimator of the parameter is proposed. It is shown that, under the null hypothesis of no change, the test statistic converges to a distribution obtained from a difference of two Brownian bridge and diverges to infinity under the epidemic alternative. Numerical results for simulation and real data example are provided.
The problem of nonparametric inference on a monotone function has been extensively studied in many particular cases. Estimators considered have often been of so-called Grenander type, being representable as the left derivative of the greatest convex minorant or least concave majorant of an estimator of a primitive function. In this paper, we provide general conditions for consistency and pointwise convergence in distribution of a class of generalized Grenander-type estimators of a monotone function. This broad class allows the minorization or majoratization operation to be performed on a data-…
In this paper, we compare maximum likelihood (ML), quasi likelihood (QL) and weighted least squares (WLS) estimators for proportional error nonlinear regression models. Literature on thermoluminescence sedimentary dating revealed another estimator similar to weighted least squares but observed responses used as weights. This estimator that we refer to as data weighted least squares (DWLS) is also included in the comparison. We show that on the order $σ, $ all four estimators behave similar to ordinary least squares estimators for standard linear regression models. On the order of $σ^2, $ the e…
This paper develops bootstrap methods to construct uniform confidence bands for nonparametric spectral estimation of Lévy densities under high-frequency observations. We assume that we observe $n$ discrete observations at frequency $1/Δ> 0$, and work with the high-frequency setup where $Δ= Δ_{n} \to 0$ and $nΔ\to \infty$ as $n \to \infty$. We employ a spectral (or Fourier-based) estimator of the Lévy density, and develop novel implementations of Gaussian multiplier (or wild) and empirical (or Efron's) bootstraps to construct confidence bands for the spectral estimator on a compact set that doe…
We study shrinkage estimation of the mean parameters of a class of multivariate distributions for which the diagonal entries of the corresponding covariance matrix are certain quadratic functions of the mean parameter. This class of distributions includes the diagonal multivariate natural exponential families. We propose two classes of semi-parametric shrinkage estimators for the mean and construct unbiased estimators of the corresponding risk. We establish the asymptotic consistency and convergence rates for these shrinkage estimators under squared error loss as both $n$, the sample size, and…
Let denote $S_n(p) = k_n^{-1} \sum_{i=1}^{k_n} \left( \log (X_{n+1-i,n} / X_{n-k_n, n}) \right)^p$, where $p > 0$, $k_n \leq n$ is a sequence of integers such that $k_n \to \infty$ and $k_n / n \to 0$, and $X_{1,n} \leq \ldots \leq X_{n,n}$ is the order statistics of iid random variables with regularly varying upper tail. The estimator $\widehat γ(n) = (S_n(p)/Γ(p+1))^{1/p}$ is an extension of the Hill estimator. We investigate the asymptotic properties of $S_n(p)$ and $\widehat γ(n)$ both for fixed $p > 0$ and for $p = p_n \to \infty$. We prove strong consistency and asymptotic normality unde…
We study the non-parametric estimation of an unknown stationary density fV of an unobserved strictly stationary volatility process $(\bm V_t)_{t\geq 0}$ on $\IRp^2 := (0,\infty)^2$ based on discrete-time observations in a stochastic volatility model. We identify the underlying multiplicative measurement error model and build an estimator based on the estimation of the Mellin transform of the scaled, integrated volatility process and a spectral cut-off regularisation of the inverse of the Mellin transform. We prove that the proposed estimator leads to a consistent estimation strategy. A fully d…
