Estimator selection has become a crucial issue in non parametric estimation. Two widely used methods are penalized empirical risk minimization (such as penalized log-likelihood estimation) or pairwise comparison (such as Lepski's method). Our aim in this paper is twofold. First we explain some general ideas about the calibration issue of estimator selection methods. We review some known results, putting the emphasis on the concept of minimal penalty which is helpful to design data-driven selection criteria. Secondly we present a new method for bandwidth selection within the framework of kernel density density estimation which is in some sense intermediate between these two main methods mentioned above. We provide some theoretical results which lead to some fully data-driven selection strategy.
We construct a density estimator and an estimator of the distribution function in the uniform deconvolution model. The estimators are based on inversion formulas and kernel estimators of the density of the observations and its derivative. Asymptotic normality and the asymptotic biases are derived.
We establish uniform-in-bandwidth consistency for kernel-type estimators of the differential entropy. We consider two kernel-type estimators of Shannon's entropy. As a consequence, an asymptotic 100% confidence interval of entropy is provided.
In this paper we study the ideal variable bandwidth kernel density estimator introduced by McKay (1993) and Jones, McKay and Hu (1994) and the plug-in practical version of the variable bandwidth kernel estimator with two sequences of bandwidths as in Giné and Sang (2013). Based on the bias and variance analysis of the ideal and true variable bandwidth kernel density estimators, we study the central limit theorems for each of them.
We consider the problem of estimating the s-th derivative of a density function f by the tilted Kernel estimator introduced in Hall and Doosti (2012). Then we further show this estimator achieves the same convergence rate, in probability, the wavelet estimators achieved as shown in Hall and Patil (1995).
We propose an estimator of the kernel-based conditional mean dependence measure obtained from an appropriate modification of a naive estimator based on usual empirical estimators. We then get asymptotic normality of this estimator both under conditional mean independence hypothesis and under the alternative hypothesis. A new test for conditional mean independence of random variables valued into Hilbert spaces is then introduced.
In this paper, we are interested in the study of beta kernel estimators from an asymptotic minimax point of view. It is well known that beta kernel estimators are, on the contrary of classical kernel estimators, "free of boundary effect" and thus are very useful in practice. The goal of this paper is to prove that there is a price to pay: for very regular functions or for certain losses, these estimators are not minimax. Nevertheless they are minimax for classical regularities such as regularity of order two or less than two, supposed commonly in the practice and for some classical losses.
We address the estimation of "extreme" conditional quantiles i.e. when their order converges to one as the sample size increases. Conditions on the rate of convergence of their order to one are provided to obtain asymptotically Gaussian distributed kernel estimators. A Weissman-type estimator and kernel estimators of the conditional tail-index are derived, permitting to estimate extreme conditional quantiles of arbitrary order.
In this paper we prove large and moderate deviations principles for the recursive kernel estimator of a probability density function and its partial derivatives. Unlike the density estimator, the derivatives estimators exhibit a quadratic behavior not only for the moderate deviations scale but also for the large deviations one. We provide results both for the pointwise and the uniform deviations.
In this paper, we prove large deviations principle for the Nadaraya-Watson estimator and for the semi-recursive kernel estimator of the regression in the multidimensional case. Under suitable conditions, we show that the rate function is a good rate function. We thus generalize the results already obtained in the unidimensional case for the Nadaraya-Watson estimator. Moreover, we give a moderate deviations principle for these two estimators. It turns out that the rate function obtained in the moderate deviations principle for the semi-recursive estimator is larger than the one obtained for the…
In one-dimensional density estimation on i.i.d. observations we suggest an adaptive cross-validation technique for the selection of a kernel estimator. This estimator is both asymptotic MISE-efficient with respect to the monotone oracle, and sharp minimax-adaptive over the whole scale of Sobolev spaces with smoothness index greater than 1/2. The proof of the central concentration inequality avoids "chaining" and relies on an additive decomposition of the empirical processes involved.
We derive asymptotic normality of kernel type deconvolution density estimators. In particular we consider deconvolution problems where the known component of the convolution has a symmetric lambda-stable distribution, 0<lambda<= 2. It turns out that the limit behavior changes if the exponent parameter lambda passes the value one, the case of Cauchy deconvolution.
Given a sample from a discretely observed compound Poisson process, we consider estimation of the density of the jump sizes. We propose a kernel type nonparametric density estimator and study its asymptotic properties. An order bound for the bias and an asymptotic expansion of the variance of the estimator are given. Pointwise weak consistency and asymptotic normality are established. The results show that, asymptotically, the estimator behaves very much like an ordinary kernel estimator.
In this paper we prove large and moderate deviations principles for the recursive kernel estimators of a probability density function defined by the stochastic approximation algorithm introduced by Mokkadem et al. [2009. The stochastic approximation method for the estimation of a probability density. J. Statist. Plann. Inference 139, 2459-2478]. We show that the estimator constructed using the stepsize which minimize the variance of the class of the recursive estimators defined in Mokkadem et al. (2009) gives the same pointwise LDP and MDP as the Rosenblatt kernel estimator. We provide results…
We derive asymptotic normality of kernel type deconvolution estimators of the density, the distribution function at a fixed point, and of the probability of an interval. We consider the so called super smooth case where the characteristic function of the known distribution decreases exponentially. It turns out that the limit behavior of the pointwise estimators of the density and distribution function is relatively straightforward while the asymptotics of the estimator of the probability of an interval depends in a complicated way on the sequence of bandwidths.
Let $θ$ and $μ$ denote the location and the size of the mode of a probability density. We study the joint convergence rates of semirecursive kernel estimators of $θ$ and $μ$. We show how the estimation of the size of the mode allows to measure the relevance of the estimation of its location. We also enlighten that, beyond their computational advantage on nonrecursive estimators, the semirecursive estimators are preferable to use for the construction on confidence regions.
We consider the asymptotic normality in $L^2$ of kernel estimators of the long run covariance kernel of stationary functional time series. Our results are established assuming a weakly dependent Bernoulli shift structure for the underlying observations, which contains most stationary functional time series models, under mild conditions. As a corollary, we obtain joint asymptotics for functional principal components computed from empirical long run covariance operators, showing that they have the favorable property of being asymptotically independent.
Averaging provides an alternative to bandwidth selection for density kernel estimation. We propose a procedure to combine linearly several kernel estimators of a density obtained from different, possibly data-driven, bandwidths. The method relies on minimizing an easily tractable approximation of the integrated square error of the combination. It provides, at a small computational cost, a final solution that improves on the initial estimators in most cases. The average estimator is proved to be asymptotically as efficient as the best possible combination (the oracle), with an error term that d…
In this paper we establish the uniform in bandwidth consistency for the transformation kernel estimator of copulas introduced in [Omelka et al.(2009)]. To this end, we first prove a uniform in bandwidth law of the iterated logarithm for the maximal deviation of this estimator from its expectation. We then show that, as n goes to infinity, the bias of the estimator converges to zero uniformly in the bandwidth h, varying over a suitable interval. A practical method of selecting the optimal bandwidth is also presented. Finally, we make conclusive simulation experiments showing the performance of …
In this paper we propose an automatic bandwidth selection of the recursive kernel density estimators with missing data in the context of global and local density estimation. We showed that, using the selected bandwidth and a special stepsize, the proposed recursive estimators outperformed the nonrecursive one in terms of estimation error in the case of global estimation. However, the recursive estimators are much better in terms of computational costs. We corroborated these theoretical results through simulation studies and on the simulated data of the Aquitaine cohort of HIV-1 infected patien…
We investigate the time-varying ARCH (tvARCH) process. It is shown that it can be used to describe the slow decay of the sample autocorrelations of the squared returns often observed in financial time series, which warrants the further study of parameter estimation methods for the model. Since the parameters are changing over time, a successful estimator needs to perform well for small samples. We propose a kernel normalized-least-squares (kernel-NLS) estimator which has a closed form, and thus outperforms the previously proposed kernel quasi-maximum likelihood (kernel-QML) estimator for small…
In this paper we consider the kernel estimators of a distribution function defined by the stochastic approximation algorithm when the observation are contamined by measurement errors. It is well known that this estimators depends heavily on the choice of a smoothing parameter called the bandwidth. We propose a specific second generation plug-in method of the deconvolution kernel distribution estimators defined by the stochastic approximation algorithm. We show that, using the proposed bandwidth selection and the stepsize which minimize the MISE (Mean Integrated Squared Error), the proposed est…
In this paper we propose an automatic selection of the bandwidth of the semi-recursive kernel estimators of a regression function defined by the stochastic approximation algorithm. We showed that, using the selected bandwidth and some special stepsizes, the proposed semi-recursive estimators will be very competitive to the nonrecursive one in terms of estimation error but much better in terms of computational costs. We corroborated these theoretical results through simulation study and a real dataset.
We address the estimation of conditional quantiles when the covariate is functional and when the order of the quantiles converges to one as the sample size increases. In a first time, we investigate to what extent these large conditional quantiles can still be estimated through a functional kernel estimator of the conditional survival function. Sufficient conditions on the rate of convergence of their order to one are provided to obtain asymptotically Gaussian distributed estimators. In a second time, basing on these result, a functional Weissman estimator is derived, permitting to estimate la…
In the Gaussian white noise model, we study the estimation of an unknown multidimensional function $f$ in the uniform norm by using kernel methods. The performances of procedures are measured by using the maxiset point of view: we determine the set of functions which are well estimated (at a prescribed rate) by each procedure. So, in this paper, we determine the maxisets associated to kernel estimators and to the Lepski procedure for the rate of convergence of the form $(\log n/n)^{\be/(2\be+d)}$. We characterize the maxisets in terms of Besov and Hölder spaces of regularity $β$.
We establish sufficient conditions for the asymptotic normality of kernel density estimators, applied to causal linear random fields. Our conditions on the coefficients of linear random fields are weaker than known results, although our assumption on the bandwidth is not minimal. The proof is based on the $m$-approximation method. As a key step, we prove a central limit theorem for triangular arrays of stationary $m$-dependent random fields with unbounded $m$. We also apply a moment inequality recently established for stationary random fields.
In this paper, we investigate a nonparametric approach to provide a recursive estimator of the transition density of a non-stationary piecewise-deterministic Markov process, from only one observation of the path within a long time. In this framework, we do not observe a Markov chain with transition kernel of interest. Fortunately, one may write the transition density of interest as the ratio of the invariant distributions of two embedded chains of the process. Our method consists in estimating these invariant measures. We state a result of consistency and a central limit theorem under some gen…
Estimating the effective dimension reduction (EDR) space, related to the semiparametric regression model introduced by Li \cite{sir}, is based on the estimation of the covariance matrix $Λ$ of the conditional expectation of the vector of predictors given the response. An estimator $\widehatΛ_n$ of $Λ$ based on kernel method was introduced by Zhu and Fang \cite{Asymptotics} who then derived, under some conditions, the asymptotic distribution of $\sqrt{n}\left(\widehatΛ_n-Λ\right)$, as $n\rightarrow +\infty$. In this paper, we obtain, under specified conditions, the almost sure convergence of $\…
We derive estimators of the density of the event times of current status data. The estimators are derived for the situations where the distribution of the observation times is known and where this distribution is unknown. The density estimators are constructed from kernel estimators of the density of transformed current status data, which have a distribution similar to uniform deconvolution data. Expansions of the expectation and variance as well as asymptotic normality are derived. A reference density based bandwidth selection method is proposed. A simulated example is presented.
We introduce a kernel estimator, to the tail index of a right-censored Pareto-type distribution, that generalizes Worms's one (Worms and Worms, 2014)in terms of weight coefficients. Under some regularity conditions, the asymptotic normality of the proposed estimator is established. In the framework of the second-order condition, we derive an asymptotically bias-reduced version to the new estimator. Through a simulation study, we conclude that one of the main features of the proposed kernel estimator is its smoothness contrary to Worms's one, which behaves, rather erratically, as a function of …
In this paper, we construct a moment inequality for mixing dependent random variables, it is of independent interest. As applications, the consistency of the kernel density estimation is investigated. Several limit theorems are established: First, the central limit theorems for the kernel density estimator $f_{n,K}(x)$ and its distribution function are constructed. Also, the convergence rates of $\|f_{n,K}(x)-Ef_{n,K}(x)\|_{p}$ in sup-norm loss and integral $L^{p}$-norm loss are proved. Moreover, the a.s. convergence rates of the supremum of $|f_{n,K}(x)-Ef_{n,K}(x)|$ over a compact set and th…
The celebrated Nadaraya-Watson kernel estimator is among the most studied method for nonparametric regression. A classical result is that its rate of convergence depends on the number of covariates and deteriorates quickly as the dimension grows, which underscores the "curse of dimensionality" and has limited its use in high dimensional settings. In this article, we show that when the true regression function is single or multi-index, the effects of the curse of dimensionality may be mitigated for the Nadaraya-Watson kernel estimator. Specifically, we prove that with $K$-fold cross-validation,…
We investigate the asymptotic behavior of the $L_p$-distance between a monotone function on a compact interval and a smooth estimator of this function. Our main result is a central limit theorem for the $L_p$-error of smooth isotonic estimators obtained by smoothing a Grenander-type estimator or isotonizing the ordinary kernel estimator. As a preliminary result we establish a similar result for ordinary kernel estimators. Our results are obtained in a general setting, which includes estimation of a monotone density, regression function and hazard rate. We also perform a simulation study for te…
Convergence rates of kernel density estimators for stationary time series are well studied. For invertible linear processes, we construct a new density estimator that converges, in the supremum norm, at the better, parametric, rate $n^{-1/2}$. Our estimator is a convolution of two different residual-based kernel estimators. We obtain in particular convergence rates for such residual-based kernel estimators; these results are of independent interest.
We investigate the asymptotic mean squared error of kernel estimators of the intensity function of a spatial point process. We show that when $n$ independent copies of a point process in $\mathbb R^d$ are superposed, the optimal bandwidth $h_n$ is of the order $n^{-1/(d+4)}$ under appropriate smoothness conditions on the kernel and true intensity function. We apply the Abramson principle to define adaptive kernel estimators and show that asymptotically the optimal adaptive bandwidth is of the order $n^{-1/(d+8)}$ under appropriate smoothness conditions.
We provide new asymptotic theory for kernel density estimators, when these are applied to autoregressive processes exhibiting moderate deviations from a unit root. This fills a gap in the existing literature, which has to date considered only nearly integrated and stationary autoregressive processes. These results have applications to nonparametric predictive regression models. In particular, we show that the null rejection probability of a nonparametric t test is controlled uniformly in the degree of persistence of the regressor. This provides a rigorous justification for the validity of the …
In this paper, we define a kernel estimator for the tail index of a Pareto-type distribution under random right-truncation and establish its asymptotic normality. A simulation study shows that, compared to the estimators recently proposed by Gardes & Stupfler (2015) and Benchaira et al. (2015), this newly introduced estimator behaves better, in terms of bias and mean squared error, for small samples.
We propose new model selection criteria based on generalized ridge estimators dominating the maximum likelihood estimator under the squared risk and the Kullback-Leibler risk in multivariate linear regression. Our model selection criteria have the following favorite properties: consistency, unbiasedness, uniformly minimum variance. Consistency is proven under an asymptotic structure $\frac{p}{n}\to c$ where $n$ is the sample size and $p$ is the parameter dimension of the response variables. In particular, our proposed class of estimators dominates the maximum likelihood estimator under the squ…
In Mombeni et al. (2019), Birnbaum-Saunders and Weibull kernel estimators were introduced for the estimation of cumulative distribution functions (c.d.f.s) supported on the half-line $[0,\infty)$. They were the first authors to use asymmetric kernels in the context of c.d.f. estimation. Their estimators were shown to perform better numerically than traditional methods such as the basic kernel method and the boundary modified version from Tenreiro (2013). In the present paper, we complement their study by introducing five new asymmetric kernel c.d.f. estimators, namely the Gamma, inverse Gamma,…
Spectral Equivariance and Geometric Transport in Reproducing Kernel Hilbert Spaces: A Unified Framework for Orthogonal Polynomial and Kernel Estimation
We study theoretically, for the first time, the Dirichlet kernel estimator introduced by Aitchison and Lauder (1985) for the estimation of multivariate densities supported on the $d$-dimensional simplex. The simplex is an important case as it is the natural domain of compositional data and has been neglected in the literature on asymmetric kernels. The Dirichlet kernel estimator, which generalizes the (non-modified) unidimensional Beta kernel estimator from Chen (1999), is free of boundary bias and non-negative everywhere on the simplex. We show that it achieves the optimal convergence rate $O…
We consider a recurrent Markov process which is an Itô semi-martingale. The Lévy kernel describes the law of its jumps. Based on observations X(0),X(Δ),...,X(nΔ), we construct an estimator for the Lévy kernel's density. We prove its consistency (as nΔ->\infty and Δ->0) and a central limit theorem. In the positive recurrent case, our estimator is asymptotically normal; in the null recurrent case, it is asymptotically mixed normal. Our estimator's rate of convergence equals the non-parametric minimax rate of smooth density estimation. The asymptotic bias and variance are analogous to those of th…
We provide new general kernel selection rules thanks to penalized least-squares criteria. We derive optimal oracle inequalities using adequate concentration tools. We also investigate the problem of minimal penalty as described in [BM07].
For the conditional mean function of panel count model with time-varying coefficients, we propose to use local kernel regression method for estimation. Partial log-likelihood with local polynomial is formed for estimation. Under some regularity conditions, strong uniform consistency rates are obtained for the local estimator. At target time point, we show that the local estimator converges in distribution to normal distribution. The baseline mean function estimator is also shown to be consistent. Simulation studies show that the time-varying coefficient estimator is close to the true value, th…
This paper considers the asymptotic behavior in $β$-Hölder spaces, and under $L^p$ losses, of a Dirichlet kernel density estimator proposed by Aitchison and Lauder (1985) for the analysis of compositional data. In recent work, Ouimet and Tolosana-Delgado (2022) established the uniform strong consistency and asymptotic normality of this estimator. As a complement, it is shown here that the Aitchison-Lauder estimator can achieve the minimax rate asymptotically for a suitable choice of bandwidth whenever $(p,β) \in [1, 3) \times (0, 2]$ or $(p, β) \in \mathcal{A}_d$, where $\mathcal{A}_d$ is a sp…
Multivariate associated kernel estimators, which depend on both target point and bandwidth matrix, are appropriate for partially or totally bounded distributions and generalize the classical ones as Gaussian. Previous studies on multivariate associated kernels have been restricted to product of univariate associated kernels, also considered having diagonal bandwidth matrices. However, it is shown in classical cases that for certain forms of target density such as multimodal, the use of full bandwidth matrices offers the potential for significantly improved density estimation. In this paper, ge…
In a previous paper, we studied a kernel estimate of the upper edge of a two-dimensional bounded set, based upon the extreme values of a Poisson point process. The initial paper "Geffroy J. (1964) Sur un problème d'estimation géométrique.Publications de l'Institut de Statistique de l'Université de Paris, XIII, 191-200" on the subject treats the frontier as the boundary of the support set for a density and the points as a random sample. We claimed in"Girard, S. and Jacob, P. (2004) Extreme values and kernel estimates of point processes boundaries.ESAIM: Probability and Statistics, 8, 150-168" t…
New local linear estimators are proposed for a wide class of nonparametric regression models. The estimators are uniformly consistent regardless of satisfying traditional conditions of depen\-dence of design elements. The estimators are the solutions of a specially weighted least-squares method. The design can be fixed or random and does not need to meet classical regularity or independence conditions. As an application, several estimators are constructed for the mean of dense functional data. The theoretical results of the study are illustrated by simulations. An example of processing real me…
Given an i.i.d sample $(Y_i,Z_i)$, taking values in $\RRR^{d'}\times \RRR^d$, we consider a collection Nadarya-Watson kernel estimators of the conditional expectations $\EEE(<c_g(z),g(Y)>+d_g(z)\mid Z=z)$, where $z$ belongs to a compact set $H\subset \RRR^d$, $g$ a Borel function on $\RRR^{d'}$ and $c_g(\cdot),d_g(\cdot)$ are continuous functions on $\RRR^d$. Given two bandwidth sequences $h_n<\wth_n$ fulfilling mild conditions, we obtain an exact and explicit almost sure limit bounds for the deviations of these estimators around their expectations, uniformly in $g\in\GG,\;z\in H$ and $h_n\le …
We propose two new kernel-type estimators of the mean residual life function $m_X(t)$ of bounded or half-bounded interval supported distributions. Though not as severe as the boundary problems in the kernel density estimation, eliminating the boundary bias problems that occur in the naive kernel estimator of the mean residual life function is needed. In this article, we utilize the property of bijective transformation. Furthermore, our proposed methods preserve the mean value property, which cannot be done by the naive kernel estimator. Some simulation results showing the estimators' performan…
Claeskens and Hjort (2003) constructed the focused information criterion (FIC) and developed frequentist model averaging methods using maximum likelihood estimators assuming the observations to be independent and identically distributed. Towards the immediate extensions and generalizations of these results, the present article is aimed at providing the focused model selection and model averaging methods using general maximum likelihood type estimators, popularly known as $M$-estimators. The necessary asymptotic theory is derived in a setup of stationary and strong mixing stochastic processes e…
