Assuming that $(X_t)_{t\in\Z}$ is a vector valued time series with a common marginal distribution admitting a density $f$, our aim is to provide a wide range of consistent estimators of $f$. We consider different methods of estimation of the density as kernel, projection or wavelets ones. Various cases of weakly dependent series are investigated including the Doukhan & Louhichi (1999)'s $η$-weak dependence condition, and the $\tilde ϕ$-dependence of Dedecker & Prieur (2005). We thus obtain results for Markov chains, dynamical systems, bilinear models, non causal Moving Average... From a moment inequality of Doukhan & Louhichi (1999), we provide convergence rates of the term of error for the estimation with the $Ł^q$ loss or almost surely, uniformly on compact subsets.
This paper provides extensions of the work on subsampling by Bertail et al. (2004) for strongly mixing case to weakly dependent case by application of the results of Doukhan and Louhichi (1999). We investigate properties of smooth and rough subsampling estimators for distributions of converging and extreme statistics when the underlying time series is η or λ-weakly dependent.
Let $X\in \mathbb{R}^p$ and $Y\in \mathbb{R}$ be two random variables. We estimate the conditional covariance matrix $\mathrm{Cov}\left(\mathrm{E}\left[\boldsymbol{X}\vert Y\right]\right)$ applying a plug-in kernel-based algorithm to its entries. Next, we investigate the estimators rate of convergence under smoothness hypotheses on the density function of $(\boldsymbol{X},Y)$. In a high-dimensional context, we improve the consistency the whole matrix estimator by providing a decreasing structure over the $\mathrm{Cov}\left(\mathrm{E}\left[\boldsymbol{X}\vert Y\right]\right)$ entries. We illust…
Convergence rates of kernel density estimators for stationary time series are well studied. For invertible linear processes, we construct a new density estimator that converges, in the supremum norm, at the better, parametric, rate $n^{-1/2}$. Our estimator is a convolution of two different residual-based kernel estimators. We obtain in particular convergence rates for such residual-based kernel estimators; these results are of independent interest.
The maximum composite likelihood estimator for parametric models of determinantal point processes (DPPs) is discussed. Since the joint intensities of these point processes are given by determinant of positive definite kernels, we have the explicit form of the joint intensities for every order. This fact enables us to consider the generalized maximum composite likelihood estimator for any order. This paper introduces the two step generalized composite likelihood estimator and shows the moment convergence of the estimator under a stationarity. Moreover, our results can yield information criteria…
Given $n$ independent random vectors with common density $f$ on $\mathbb{R}^d$, we study the weak convergence of three empirical-measure based estimators of the convex $λ$-level set $L_λ$ of $f$, namely the excess mass set, the minimum volume set and the maximum probability set, all selected from a class of convex sets $\mathcal{A}$ that contains $L_λ$. Since these set-valued estimators approach $L_λ$, even the formulation of their weak convergence is non-standard. We identify the joint limiting distribution of the symmetric difference of $L_λ$ and each of the three estimators, at rate $n^{-1/…
In this contribution we introduce weakly locally stationary time series through the local approximation of the non-stationary covariance structure by a stationary one. This allows us to define autoregression coefficients in a non-stationary context, which, in the particular case of a locally stationary Time Varying Autoregressive (TVAR) process, coincide with the generating coefficients. We provide and study an estimator of the time varying autoregression coefficients in a general setting. The proposed estimator of these coefficients enjoys an optimal minimax convergence rate under limited smo…
It has been shown in literature that the Lasso estimator, or l1-penalized least squares estimator, enjoys good oracle properties. This paper examines which special properties of the l1-penalty allow for sharp oracle results, and then extends the situation to general norm-based penalties that satisfy a weak decomposability condition.
We give a thorough description of the asymptotic property of the maximum likelihood estimator (MLE) of the skewness parameter of a Skew Brownian Motion (SBM). Thanks to recent results on the Central Limit Theorem of the rate of convergence of estimators for the SBM, we prove a conjecture left open that the MLE has asymptotically a mixed normal distribution involving the local time with a rate of convergence of order $1/4$. We also give a series expansion of the MLE and study the asymptotic behavior of the score and its derivatives, as well as their variation with the skewness parameter. In par…
In this paper, we investigate the almost sure convergence, in supremum norm, of the rank-based linear wavelet estimator for a multivariate copula density. Based on empirical process tools, we prove a uniform limit law for the deviation, from its expectation, of an oracle estimator (obtained for known margins), from which we derive the exact convergence rate of the rank-based linear estimator. This rate reveals to be optimal in a minimax sense over Besov balls for the supremum norm loss, whenever the resolution level is suitably chosen.
We find upper bounds for the probability of underestimation and overestimation errors in penalized likelihood context tree estimation. The bounds are explicit and applies to processes of not necessarily finite memory. We allow for general penalizing terms and we give conditions over the maximal depth of the estimated trees in order to get strongly consistent estimates. This generalizes previous results obtained in the case of estimation of the order of a Markov chain.
The problem to establish not only the asymptotic distribution results for statistical estimators but also the moment convergence of the estimators has been recognized as an important issue in advanced theories of statistics. One of the main goals of this paper is to present a metod to derive the moment convergence of $Z$-estimators as it has been done for $M$-estimators. Another goal of this paper is to develop a general, unified approach, based on some partial estimation functions which we call "$Z$-process", to the change point problems for ergodic models as well as some models where the Fis…
We consider the non-parametric Poisson regression problem where the integer valued response $Y$ is the realization of a Poisson random variable with parameter $λ(X)$. The aim is to estimate the functional parameter $λ$ from independent or weakly dependent observations $(X_1,Y_1),\ldots,(X_n,Y_n)$ in a random design framework. First we determine upper risk bounds for projection estimators on finite dimensional subspaces under mild conditions. In the case of Sobolev ellipsoids the obtained rates of convergence turn out to be optimal. The main part of the paper is devoted to the construction of a…
In this paper we study some asymptotic properties of the kernel conditional quantile estimator with randomly left-truncated data which exhibit some kind of dependence. We extend the result obtained by Lemdani, Ould-Saïd and Poulin [16] in the iid case. The uniform strong convergence rate of the estimator under strong mixing hypothesis is obtained.
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
This paper studies the asymptotic behavior of penalized spline estimates of derivatives. In particular, we show that simply differentiating the penalized spline estimator of the mean regression function itself to estimate the corresponding derivative achieves the optimal L2 rate of convergence.
We provide general formulation of weak identification in semiparametric models and an efficiency concept. Weak identification occurs when a parameter is weakly regular, i.e., when it is locally homogeneous of degree zero. When this happens, consistent or equivariant estimation is shown to be impossible. We then show that there exists an underlying regular parameter that fully characterizes the weakly regular parameter. While this parameter is not unique, concepts of sufficiency and minimality help pin down a desirable one. If estimation of minimal sufficient underlying parameters is inefficien…
We study the uniform convergence rate of the nonparametric maximum likelihood estimator (MLE) for the sub-distribution functions in the current status data with competing risks model. It is known that the MLE have $L^2$-norm convergence rate $O_P(n^{-1/3})$ in the absolutely continuous case, but there is no arguments for the same rate of uniform convergence. We specify conditions for the uniform convergence rate $O_P(n^{-1/3}\log^{1/3} n)$ of the MLE for the sub-distribution functions of competing risks on finite intervals. The obtained result refines known uniform convergence rate in the part…
Gradient information on the sampling distribution can be used to reduce the variance of Monte Carlo estimators via Stein's method. An important application is that of estimating an expectation of a test function along the sample path of a Markov chain, where gradient information enables convergence rate improvement at the cost of a linear system which must be solved. The contribution of this paper is to establish theoretical bounds on convergence rates for a class of estimators based on Stein's method. Our analysis accounts for (i) the degree of smoothness of the sampling distribution and test…
The extremes of a stationary time series typically occur in clusters. A primary measure for this phenomenon is the extremal index, representing the reciprocal of the expected cluster size. Both a disjoint and a sliding blocks estimator for the extremal index are analyzed in detail. In contrast to many competitors, the estimators only depend on the choice of one parameter sequence. We derive an asymptotic expansion, prove asymptotic normality and show consistency of an estimator for the asymptotic variance. Explicit calculations in certain models and a finite-sample Monte Carlo simulation study…
Support size estimation and the related problem of unseen species estimation have wide applications in ecology and database analysis. Perhaps the most used support size estimator is the Chao estimator. Despite its wide spread use, little is known about its theoretical properties. We analyze the Chao estimator and show that its worst case mean squared error (MSE) is smaller than the MSE of the plug-in estimator by a factor of $\mathcal{O} ((k/n)^4)$, where $k$ is the maximum support size and $n$ is the number of samples. Our main technical contribution is a new method to analyze rational estima…
Under the assumption that the true density is decreasing, it is well known that the Grenander estimator converges at rate $n^{1/3}$ if the true density is curved [Sankhyā Ser. A 31 (1969) 23-36] and at rate $n^{1/2}$ if the density is flat [Ann. Probab. 11 (1983) 328-345; Canad. J. Statist. 27 (1999) 557-566]. In the case that the true density is misspecified, the results of Patilea [Ann. Statist. 29 (2001) 94-123] tell us that the global convergence rate is of order $n^{1/3}$ in Hellinger distance. Here, we show that the local convergence rate is $n^{1/2}$ at a point where the density is miss…
We derive the $l_{\infty}$ convergence rate simultaneously for Lasso and Dantzig estimators in a high-dimensional linear regression model under a mutual coherence assumption on the Gram matrix of the design and two different assumptions on the noise: Gaussian noise and general noise with finite variance. Then we prove that simultaneously the thresholded Lasso and Dantzig estimators with a proper choice of the threshold enjoy a sign concentration property provided that the non-zero components of the target vector are not too small.
A number of discrete time, finite population size models in genetics describing the dynamics of allele frequencies are known to converge (subject to suitable scaling) to a diffusion process in the infinite population limit, termed the Wright-Fisher diffusion. In this article we show that the diffusion is ergodic uniformly in the selection and mutation parameters, and that the measures induced by the solution to the stochastic differential equation are uniformly locally asymptotically normal. Subsequently these two results are used to analyse the statistical properties of the Maximum Likelihood…
We consider a general monotone regression estimation where we allow for independent and dependent regressors. We propose a modification of the classical isotonic least squares estimator and establish its rate of convergence for the integrated $L_1$-loss function. The methodology captures the shape of the data without assuming additivity or a parametric form for the regression function. Furthermore, the degree of smoothing is chosen automatically and no auxiliary tuning is required for the theoretical analysis. Some simulations and two real data illustrations complement the study of the propose…
In this work we construct an optimal linear shrinkage estimator for the covariance matrix in high dimensions. The recent results from the random matrix theory allow us to find the asymptotic deterministic equivalents of the optimal shrinkage intensities and estimate them consistently. The developed distribution-free estimators obey almost surely the smallest Frobenius loss over all linear shrinkage estimators for the covariance matrix. The case we consider includes the number of variables $p\rightarrow\infty$ and the sample size $n\rightarrow\infty$ so that $p/n\rightarrow c\in (0, +\infty)$. …
The paper studies large sample asymptotic properties of the Maximum Likelihood Estimator (MLE) for the parameter of a continuous time Markov chain, observed in white noise. Using the method of weak convergence of likelihoods due to I.Ibragimov and R.Khasminskii, consistency, asymptotic normality and convergence of moments are established for MLE under certain strong ergodicity conditions of the chain.
This paper generalizes asymptotic properties obtained in the observation-driven times series models considered by \cite{dou:kou:mou:2013} in the sense that the conditional law of each observation is also permitted to depend on the parameter. The existence of ergodic solutions and the consistency of the Maximum Likelihood Estimator (MLE) are derived under easy-to-check conditions. The obtained conditions appear to apply for a wide class of models. We illustrate our results with specific observation-driven times series, including the recently introduced NBIN-GARCH and NM-GARCH models, demonstrat…
We show that spline and wavelet series regression estimators for weakly dependent regressors attain the optimal uniform (i.e. sup-norm) convergence rate $(n/\log n)^{-p/(2p+d)}$ of Stone (1982), where $d$ is the number of regressors and $p$ is the smoothness of the regression function. The optimal rate is achieved even for heavy-tailed martingale difference errors with finite $(2+(d/p))$th absolute moment for $d/p<2$. We also establish the asymptotic normality of t statistics for possibly nonlinear, irregular functionals of the conditional mean function under weak conditions. The results are p…
We consider a nonparametric regression model $Y=r(X)+\varepsilon$ with a random covariate $X$ that is independent of the error $\varepsilon$. Then the density of the response $Y$ is a convolution of the densities of $\varepsilon$ and $r(X)$. It can therefore be estimated by a convolution of kernel estimators for these two densities, or more generally by a local von Mises statistic. If the regression function has a nowhere vanishing derivative, then the convolution estimator converges at a parametric rate. We show that the convergence holds uniformly, and that the corresponding process obeys a …
In this paper, we prove $L_p,\ p\geq 2$ and almost sure convergence of tail index estimator mentioned in \cite{grama2008} under random censoring and several assumptions. $p$th moment of the error of the estimator is proved to be of order $O\left(\frac{1}{\log^{mκ/2}n}\right)$ with given assumptions. We also perform several finite sample simulations to quantify performance of this estimator. Finite sample results show that the proposed estimator is effective in finding underlying tail index even when censor rate is high.
Optimal transport maps between two probability distributions $μ$ and $ν$ on $\mathbb{R}^d$ have found extensive applications in both machine learning and statistics. In practice, these maps need to be estimated from data sampled according to $μ$ and $ν$. Plug-in estimators are perhaps most popular in estimating transport maps in the field of computational optimal transport. In this paper, we provide a comprehensive analysis of the rates of convergences for general plug-in estimators defined via barycentric projections. Our main contribution is a new stability estimate for barycentric projectio…
This paper considers the nonparametric regression model with negatively super-additive dependent (NSD) noise and investigates the convergence rates of thresholding estimators. It is shown that the term-by-term thresholding estimator achieves nearly optimal and the block thresholding estimator attains optimal (or nearly optimal) convergence rates over Besov spaces. Additionally, some numerical simulations are implemented to substantiate the validity and adaptivity of the thresholding estimators with the presence of NSD noise.
This paper develops a general theory on rates of convergence of penalized spline estimators for function estimation when the likelihood functional is concave in candidate functions, where the likelihood is interpreted in a broad sense that includes conditional likelihood, quasi-likelihood, and pseudo-likelihood. The theory allows all feasible combinations of the spline degree, the penalty order, and the smoothness of the unknown functions. According to this theory, the asymptotic behaviors of the penalized spline estimators depends on interplay between the spline knot number and the penalty pa…
We study kmeans clustering estimation of panel data models with a latent group structure and $N$ units and $T$ time periods under long panel asymptotics. We show that the group-specific coefficients can be estimated at the parametric root $NT$ rate even if error variances diverge as $T \to \infty$ and some units are asymptotically misclassified. This limit case approximates empirically relevant settings and is not covered by existing asymptotic results.
A new nonparametric estimator of a convex regression function in any dimension is proposed and its convergence properties are studied. We start by using any estimator of the regression function and we \emph{convexify} it by taking the convex envelope of a sample of the approximation obtained. We prove that the uniform rate of convergence of the estimator is maintained after the convexification is applied. The finite sample properties of the new estimator are investigated by means of a simulation study and the application of the new method is demonstrated in examples.
When the study variable is functional and storage capacities are limited or transmission costs are high, selecting with survey sampling techniques a small fraction of the observations is an interesting alternative to signal compression techniques, particularly when the goal is the estimation of simple quantities such as means or totals. We extend, in this functional framework, model-assisted estimators with linear regression models that can take account of auxiliary variables whose totals over the population are known. We first show, under weak hypotheses on the sampling design and the regular…
The geometric median, also called $L^{1}$-median, is often used in robust statistics. Moreover, it is more and more usual to deal with large samples taking values in high dimensional spaces. In this context, a fast recursive estimator has been introduced by Cardot, Cenac and Zitt. This work aims at studying more precisely the asymptotic behavior of the estimators of the geometric median based on such non linear stochastic gradient algorithms. The $L^{p}$ rates of convergence as well as almost sure rates of convergence of these estimators are derived in general separable Hilbert spaces. Moreove…
We derive process limit distribution results for the Nelson-Aalen estimator of a hasard function and for the Kaplan-Meier estimator of a distribution function, under different dependence assumptions. The data are assumed to be right censored observations of a stationary time series. We treat weakly dependent as well as long range dependent data, and allow for qualitative differences in the dependence for the censoring times versus the time of interest
Crowdsourcing has become a primary means for label collection in many real-world machine learning applications. A classical method for inferring the true labels from the noisy labels provided by crowdsourcing workers is Dawid-Skene estimator. In this paper, we prove convergence rates of a projected EM algorithm for the Dawid-Skene estimator. The revealed exponent in the rate of convergence is shown to be optimal via a lower bound argument. Our work resolves the long standing issue of whether Dawid-Skene estimator has sound theoretical guarantees besides its good performance observed in practic…
This paper addresses the problem of regression to reconstruct functions, which are observed with superimposed errors at random locations. We address the problem in reproducing kernel Hilbert spaces. It is demonstrated that the estimator, which is often derived by employing Gaussian random fields, converges in the mean norm of the reproducing kernel Hilbert space to the conditional expectation and this implies local and uniform convergence of this function estimator. By preselecting the kernel, the problem does not suffer from the curse of dimensionality. The paper analyzes the statistical prop…
Let $(X_i)_{i=1,...,n}$ be a possibly nonstationary sequence such that $\mathscr{L}(X_i)=P_n$ if $i\leq nθ$ and $\mathscr{L}(X_i)=Q_n$ if $i>nθ$, where $0<θ<1$ is the location of the change-point to be estimated. We construct a class of estimators based on the empirical measures and a seminorm on the space of measures defined through a family of functions $\mathcal{F}$. We prove the consistency of the estimator and give rates of convergence under very general conditions. In particular, the $1/n$ rate is achieved for a wide class of processes including long-range dependent sequences and even no…
For the problem of nonparametric estimation of signal in Gaussian noise we point out the strong asymptotically minimax estimators on maxisets for linear estimators (see \cite{ker93,rio}). It turns out that the order of rates of convergence of Pinsker estimator on this maxisets is worse than the order of rates of convergence for the class of linear estimators considered on this maxisets. We show that balls in Sobolev spaces are maxisets for Pinsker estimators.
Zou [J. Amer. Statist. Assoc. 101 (2006) 1418-1429] proposed the Adaptive LASSO (ALASSO) method for simultaneous variable selection and estimation of the regression parameters, and established its oracle property. In this paper, we investigate the rate of convergence of the ALASSO estimator to the oracle distribution when the dimension of the regression parameters may grow to infinity with the sample size. It is shown that the rate critically depends on the choices of the penalty parameter and the initial estimator, among other factors, and that confidence intervals (CIs) based on the oracle l…
