In this paper, we are interested in the problem of smoothing parameter selection in nonparametric curve estimation under dependent errors. We focus on kernel estimation and the case when the errors form a general stationary sequence of martingale difference random variables where neither linearity assumption nor "all moments are finite" are required.We compare the behaviors of the smoothing bandwidths obtained by minimizing either the unknown average squared error, the theoretical mean average squared error, a Mallows-type criterion adapted to the dependent case and the family of criteria known as generalized cross validation (GCV) extensions of the Mallows' criterion. We prove that these three minimizers and those based on the GCV family are first-order equivalent in probability. We give also a normal asymptotic behavior of the gap between the minimizer of the average square error and that of the Mallows-type criterion. This is extended to the GCV family.Finally, we apply our theoretical results to a specific case of martingale difference sequence, namely the Auto-Regressive Conditional Heteroscedastic (ARCH(1)) process.A Monte-carlo simulation study, for this regression model with ARCH(1) process, is conducted.
We define a new bandwidth-dependent kernel density estimator that improves existing convergence rates for the bias, and preserves that of the variation, when the error is measured in $L_1$. No additional assumptions are imposed to the extant literature.
We consider the problem of adaptive estimation of the regression function in a framework where we replace ergodicity assumptions (such as independence or mixing) by another structural assumption on the model. Namely, we propose adaptive upper bounds for kernel estimators with data-driven bandwidth (Lepski's selection rule) in a regression model where the noise is an increment of martingale. It includes, as very particular cases, the usual i.i.d. regression and auto-regressive models. The cornerstone tool for this study is a new result for self-normalized martingales, called ``stability'', whic…
Bandwidth selection is crucial in the kernel estimation of density level sets. A risk based on the symmetric difference between the estimated and true level sets is usually used to measure their proximity. In this paper we provide an asymptotic $L^p$ approximation to this risk, where $p$ is characterized by the weight function in the risk. In particular the excess risk corresponds to an $L^2$ type of risk, and is adopted to derive an optimal bandwidth for nonparametric level set estimation of $d$-dimensional density functions ($d\geq 1$). A direct plug-in bandwidth selector is developed for ke…
It is shown that, for kernel-based classification with univariate distributions and two populations, optimal bandwidth choice has a dichotomous character. If the two densities cross at just one point, where their curvatures have the same signs, then minimum Bayes risk is achieved using bandwidths which are an order of magnitude larger than those which minimize pointwise estimation error. On the other hand, if the curvature signs are different, or if there are multiple crossing points, then bandwidths of conventional size are generally appropriate. The range of different modes of behavior is na…
We present a greedy method for simultaneously performing local bandwidth selection and variable selection in nonparametric regression. The method starts with a local linear estimator with large bandwidths, and incrementally decreases the bandwidth of variables for which the gradient of the estimator with respect to bandwidth is large. The method--called rodeo (regularization of derivative expectation operator)--conducts a sequence of hypothesis tests to threshold derivatives, and is easy to implement. Under certain assumptions on the regression function and sampling density, it is shown that t…
We present a greedy method for simultaneously performing local bandwidth selection and variable selection in nonparametric regression. The method starts with a local linear estimator with large bandwidths, and incrementally decreases the bandwidth of variables for which the gradient of the estimator with respect to bandwidth is large. The method--called rodeo (regularization of derivative expectation operator)--conducts a sequence of hypothesis tests to threshold derivatives, and is easy to implement. Under certain assumptions on the regression function and sampling density, it is shown that t…
In the common nonparametric regression model the problem of testing for a specific parametric form of the variance function is considered. Recently Dette and Hetzler (2008) proposed a test statistic, which is based on an empirical process of pseudo residuals. The process converges weakly to a Gaussian process with a complicated covariance kernel depending on the data generating process. In the present paper we consider a standardized version of this process and propose a martingale transform to obtain asymptotically distribution free tests for the corresponding Kolmogorov-Smirnov and Cramér-vo…
We consider the nonparametric estimation of the density function of weakly and strongly dependent processes with noisy observations. We show that in the ordinary smooth case the optimal bandwidth choice can be influenced by long range dependence, as opposite to the standard case, when no noise is present. In particular, if the dependence is moderate the bandwidth, the rates of mean-square convergence and, additionally, central limit theorem are the same as in the i.i.d. case. If the dependence is strong enough, then the bandwidth choice is influenced by the strength of dependence, which is dif…
Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstoc…
For nonparametric regression with one-sided errors and a boundary curve model for Poisson point processes we consider the problem of efficient estimation for linear functionals. The minimax optimal rate is obtained by an unbiased estimation method which nevertheless depends on a Hölder condition or monotonicity assumption for the underlying regression or boundary function. We first construct a simple blockwise estimator and then build up a nonparametric maximum-likelihood approach for exponential noise variables and the point process model. In that approach also non-asymptotic efficiency is ob…
We investigate the nonparametric estimation for regression in a fixed-design setting when the errors are given by a field of dependent random variables. Sufficient conditions for kernel estimators to converge uniformly are obtained. These estimators can attain the optimal rates of uniform convergence and the results apply to a large class of random fields which contains martingale-difference random fields and mixing random fields.
This work develops change-point methods for statistics of high-frequency data. The main interest is in the volatility of an Itô semi-martingale, the latter being discretely observed over a fixed time horizon. We construct a minimax-optimal test to discriminate continuous paths from paths comprising volatility jumps. This is embedded into a more general theory to infer the smoothness of volatilities. In a high-frequency framework we prove weak convergence of the test statistic under the hypothesis to an extreme value distribution. Moreover, we develop methods to infer changes in the Hurst param…
We provide a nonparametric method for the computation of instantaneous multivariate volatility for continuous semi-martingales, which is based on Fourier analysis. The co-volatility is reconstructed as a stochastic function of time by establishing a connection between the Fourier transform of the prices process and the Fourier transform of the co-volatility process. A nonparametric estimator is derived given a discrete unevenly spaced and asynchronously sampled observations of the asset price processes. The asymptotic properties of the random estimator are studied: namely, consistency in proba…
We investigate and extend the conformal prediction method due to Vovk,Gammerman and Shafer (2005) to construct nonparametric prediction regions. These regions have guaranteed distribution free, finite sample coverage, without any assumptions on the distribution or the bandwidth. Explicit convergence rates of the loss function are established for such regions under standard regularity conditions. Approximations for simplifying implementation and data driven bandwidth selection methods are also discussed. The theoretical properties of our method are demonstrated through simulations.
In a functional setting, we propose two test statistics to highlight the Poisson nature of a Cox process when n copies of the process are available. Our approach involves a comparison of the empirical mean and the empirical variance of the functional data and can be seen as an extended version of a classical overdispersion test for counting data. The limiting distributions of our statistics are derived using a functional central limit theorem for c`adl`ag martingales. We also study the asymptotic power of our tests under some local alternatives. Our procedure is easily implementable and does n…
This paper deals with the nonparametric density estimation of the regression error term assuming its independence with the covariate. The difference between the feasible estimator which uses the estimated residuals and the unfeasible one using the true residuals is studied. An optimal choice of the bandwidth used to estimate the residuals is given. We also study the asymptotic normality of the feasible kernel estimator and its rate-optimality.
We introduce a nonparametric way to estimate the global probability density function for a random persistence diagram. Precisely, a kernel density function centered at a given persistence diagram and a given bandwidth is constructed. Our approach encapsulates the number of topological features and considers the appearance or disappearance of features near the diagonal in a stable fashion. In particular, the structure of our kernel individually tracks long persistence features, while considering features near the diagonal as a collective unit. The choice to describe short persistence features a…
We propose nonparametric estimation of divergence measures between continuous distributions. Our approach is based on a plug-in kernel- type estimators of density functions. We give the uniform in bandwidth consistency for the proposal estimators. As a consequence, their asymp- totic 100% confidence intervals are also provided.
We discuss nonparametric tests for parametric specifications of regression quantiles. The test is based on the comparison of parametric and nonparametric fits of these quantiles. The nonparametric fit is a Nadaraya-Watson quantile smoothing estimator. An asymptotic treatment of the test statistic requires the development of new mathematical arguments. An approach that makes only use of plugging in a Bahadur expansion of the nonparametric estimator is not satisfactory. It requires too strong conditions on the dimension and the choice of the bandwidth. Our alternative mathematical approach requi…
In the context of estimating local modes of a conditional density based on kernel density estimators, we show that existing bandwidth selection methods developed for kernel density estimation are unsuitable for mode estimation. We propose two methods to select bandwidths tailored for mode estimation in the regression setting. Numerical studies using synthetic data and a real-life data set are carried out to demonstrate the performance of the proposed methods in comparison with several well received bandwidth selection methods for density estimation.
A two-class mixture model, where the density of one of the components is known, is considered. We address the issue of the nonparametric adaptive estimation of the unknown probability density of the second component. We propose a randomly weighted kernel estimator with a fully data-driven bandwidth selection method, in the spirit of the Goldenshluger and Lepski method. An oracle-type inequality for the pointwise quadratic risk is derived as well as convergence rates over Holder smoothness classes. The theoretical results are illustrated by numerical simulations.
The celebrated Nadaraya-Watson kernel estimator is among the most studied method for nonparametric regression. A classical result is that its rate of convergence depends on the number of covariates and deteriorates quickly as the dimension grows, which underscores the "curse of dimensionality" and has limited its use in high dimensional settings. In this article, we show that when the true regression function is single or multi-index, the effects of the curse of dimensionality may be mitigated for the Nadaraya-Watson kernel estimator. Specifically, we prove that with $K$-fold cross-validation,…
We derive estimators of the density of the event times of current status data. The estimators are derived for the situations where the distribution of the observation times is known and where this distribution is unknown. The density estimators are constructed from kernel estimators of the density of transformed current status data, which have a distribution similar to uniform deconvolution data. Expansions of the expectation and variance as well as asymptotic normality are derived. A reference density based bandwidth selection method is proposed. A simulated example is presented.
This article deals with adaptive nonparametric estimation for Lévy processes observed at low frequency. For general linear functionals of the Lévy measure, we construct kernel estimators, provide upper risk bounds and derive rates of convergence under regularity assumptions. Our focus lies on the adaptive choice of the bandwidth, using model selection techniques. We face here a non-standard problem of model selection with unknown variance. A new approach towards this problem is proposed, which also allows a straightforward generalization to a classical density deconvolution framework.
Consider a Gaussian nonparametric regression problem having both an unknown mean function and unknown variance function. This article presents a class of difference-based kernel estimators for the variance function. Optimal convergence rates that are uniform over broad functional classes and bandwidths are fully characterized, and asymptotic normality is also established. We also show that for suitable asymptotic formulations our estimators achieve the minimax rate.
We investigate the behavior of the Generalized Likelihood Ratio Test (GLRT) (Fan, Zhang and Zhang [Ann. Statist. 29 (2001) 153-193]) for time varying coefficient models where the regressors and errors are non-stationary time series and can be cross correlated. It is found that the GLRT retains the minimax rate of local alternative detection under weak dependence and non-stationarity. However, in general, the Wilks phenomenon as well as the classic residual bootstrap are sensitive to either conditional heteroscedasticity of the errors, non-stationarity or temporal dependence. An averaged test i…
Consider the nonparametric regression model Y=m(X)+E, where the function m is smooth but unknown, and E is independent of X. An estimator of the density of the error term E is proposed and its weak consistency is obtained. The contribution of this paper is twofold. First, we evaluate the impact of the estimation of the regression function on the error density estimator. Secondly, the optimal choices of the first and second step bandwidths used for estimating the regression function and the error density are proposed. Further, we investigate the asymptotic normality of the error density estimat…
We consider a stochastic individual-based model in continuous time to describe a size-structured population for cell divisions. This model is motivated by the detection of cellular aging in biology. We address here the problem of nonparametric estimation of the kernel ruling the divisions based on the eigenvalue problem related to the asymptotic behavior in large population. This inverse problem involves a multiplicative deconvolution operator. Using Fourier technics we derive a nonparametric estimator whose consistency is studied. The main difficulty comes from the non-standard equations conn…
Surface integrals on density level sets often appear in asymptotic results in nonparametric level set estimation (such as for confidence regions and bandwidth selection). Also surface integrals can be used to describe the shape of level sets (using such as Willmore energy and Minkowski functionals), and link geometry (curvature) and topology (Euler characteristic) of level sets through the Gauss-Bonnet theorem. We consider three estimators of surface integrals on density level sets, one as a direct plug-in estimator, and the other two based on different neighborhoods of level sets. We allow th…
We derive a nonparametric test for constant beta over a fixed time interval from high-frequency observations of a bivariate \Ito semimartingale. Beta is defined as the ratio of the spot continuous covariation between an asset and a risk factor and the spot continuous variation of the latter. The test is based on the asymptotic behavior of the covariation between the risk factor and an estimate of the residual component of the asset, that is orthogonal (in martingale sense) to the risk factor, over blocks with asymptotically shrinking time span. Rate optimality of the test over smoothness class…
Nonparametric methods play a central role in modern empirical work. While they provide inference procedures that are more robust to parametric misspecification bias, they may be quite sensitive to tuning parameter choices. We study the effects of bias correction on confidence interval coverage in the context of kernel density and local polynomial regression estimation, and prove that bias correction can be preferred to undersmoothing for minimizing coverage error and increasing robustness to tuning parameter choice. This is achieved using a novel, yet simple, Studentization, which leads to a n…
A piecewise-deterministic Markov process is a stochastic process whose behavior is governed by an ordinary differential equation punctuated by random jumps occurring at random times. We focus on the nonparametric estimation problem of the jump rate for such a stochastic model observed within a long time interval under an ergodicity condition. We introduce an uncountable class (indexed by the deterministic flow) of recursive kernel estimates of the jump rate and we establish their strong pointwise consistency as well as their asymptotic normality. We propose to choose among this class the estim…
This paper discusses the local linear smoothing to estimate the unknown first and second infinitesimal moments in second-order jump-diffusion model based on Gamma asymmetric kernels. Under the mild conditions, we obtain the weak consistency and the asymptotic normality of these estimators for both interior and boundary design points. Besides the standard properties of the local linear estimation such as simple bias representation and boundary bias correction, the local linear smoothing using Gamma asymmetric kernels possess some extra advantages such as variable bandwidth, variance reduction a…
We propose completely nonparametric methodology to investigate location-scale modelling of two-component mixture cure models, where the responses of interest are only indirectly observable due to the presence of censoring and the presence of so-called long-term survivors that are always censored. We use covariate-localized nonparametric estimators, which depend on a bandwidth sequence, to propose an estimator of the error distribution function that has not been considered before in the literature. When this bandwidth belongs to a certain range of undersmoothing bandwidths, the asymptotic distr…
Suppose that $Y = ψ(X_1, \ldots, X_p)$, where $(X_1,\ldots, X_p)^\top$ are random inputs, $Y$ is the output, and $ψ(\cdot)$ is an unknown link function. The Sobol indices gauge the sensitivity of each $X$ against $Y$ by estimating the regression curve's variability between them. In this paper, we estimate these curves with a kernel-based method. The method allows to estimate the first order indices when the link between the independent and dependent variables is unknown. The kernel-based methods need a bandwidth to average the observations. For finite samples, the cross-validation method is fa…
In this paper, we propose to construct confidence bands by bootstrapping the debiased kernel density estimator (for density estimation) and the debiased local polynomial regression estimator (for regression analysis). The idea of using a debiased estimator was recently employed by Calonico et al. (2018b) to construct a confidence interval of the density function (and regression function) at a given point by explicitly estimating stochastic variations. We extend their ideas of using the debiased estimator and further propose a bootstrap approach for constructing simultaneous confidence bands. T…
A confidence sequence is a sequence of confidence intervals that is uniformly valid over an unbounded time horizon. Our work develops confidence sequences whose widths go to zero, with nonasymptotic coverage guarantees under nonparametric conditions. We draw connections between the Cramér-Chernoff method for exponential concentration, the law of the iterated logarithm (LIL), and the sequential probability ratio test -- our confidence sequences are time-uniform extensions of the first; provide tight, nonasymptotic characterizations of the second; and generalize the third to nonparametric settin…
This paper deals with the nonparametric estimation in heteroscedastic regression $ Y_i=f(X_i)+ξ_i, \: i=1,...,n $, with incomplete information, i.e. each real random variable $ ξ_i $ has a density $ g_{i} $ which is unknown to the statistician. The aim is to estimate the regression function $ f $ at a given point. Using a local polynomial fitting from M-estimator denoted $ \hat f^h $ and applying Lepski's procedure for the bandwidth selection, we construct an estimator $ \hat f^{\hat h} $ which is adaptive over the collection of isotropic Hölder classes. In particular, we establish new exponen…
We consider the problem of constructing honest confidence intervals (CIs) for a scalar parameter of interest, such as the regression discontinuity parameter, in nonparametric regression based on kernel or local polynomial estimators. To ensure that our CIs are honest, we use critical values that take into account the possible bias of the estimator upon which the CIs are based. We show that this approach leads to CIs that are more efficient than conventional CIs that achieve coverage by undersmoothing or subtracting an estimate of the bias. We give sharp efficiency bounds of using different ker…
Carnegie Mellon University（卡内基梅隆大学）；University of Rome（罗马大学）
We derive nonparametric confidence intervals for the eigenvalues of the Hessian at modes of a density estimate. This provides information about the strength and shape of modes and can also be used as a significance test. We use a data-splitting approach in which potential modes are identified using the first half of the data and inference is done with the second half of the data. To get valid confidence sets for the eigenvalues, we use a bootstrap based on an elementary-symmetric-polynomial (ESP) transformation. This leads to valid bootstrap confidence sets regardless of any multiplicities in …
The problem of estimating the regression function in a fixed design models with correlated observations is considered. Such observations are obtained from several experimental units, each of them forms a time series. Based on the trapezoidal rule, we propose a simple kernel estimator and we derive the asymptotic expression of its integrated mean squared error IMSE and its asymptotic normality. The problems of the optimal bandwidth and the optimal design with respect to the asymptotic IMSE are also investigated. Finally, a simulation study is conducted to study the performance of the new estima…
Important information concerning a multivariate data set, such as clusters and modal regions, is contained in the derivatives of the probability density function. Despite this importance, nonparametric estimation of higher order derivatives of the density functions have received only relatively scant attention. Kernel estimators of density functions are widely used as they exhibit excellent theoretical and practical properties, though their generalization to density derivatives has progressed more slowly due to the mathematical intractabilities encountered in the crucial problem of bandwidth (…
Reconstruction of sets from a random sample of points intimately related to them is the goal of set estimation theory. Within this context, a particular problem is the one related with the reconstruction of density level sets and specifically, those ones with a high probability content, namely highest density regions. We define highest density regions for directional data and provide a plug-in estimator, based on kernel smoothing. A suitable bootstrap bandwidth selector is provided for the practical implementation of the proposal. An extensive simulation study shows the performance of the plug…
University of California, San Diego（加州大学圣地亚哥分校）；University of California, Berkeley（加州大学伯克利分校）
We propose a new algorithmic framework for sequential hypothesis testing with i.i.d. data, which includes A/B testing, nonparametric two-sample testing, and independence testing as special cases. It is novel in several ways: (a) it takes linear time and constant space to compute on the fly, (b) it has the same power guarantee as a non-sequential version of the test with the same computational constraints up to a small factor, and (c) it accesses only as many samples as are required - its stopping time adapts to the unknown difficulty of the problem. All our test statistics are constructed to b…
Carnegie Mellon University（卡内基梅隆大学）；University of Chicago（芝加哥大学）
We present algorithms for nonparametric regression in settings where the data are obtained sequentially. While traditional estimators select bandwidths that depend upon the sample size, for sequential data the effective sample size is dynamically changing. We propose a linear time algorithm that adjusts the bandwidth for each new data point, and show that the estimator achieves the optimal minimax rate of convergence. We also propose the use of online expert mixing algorithms to adapt to unknown smoothness of the regression function. We provide simulations that confirm the theoretical results,…
This thesis deals with the nonparametric estimation of density f of the regression error term E of the model Y=m(X)+E, assuming its independence with the covariate X. The difficulty linked to this study is the fact that the regression error E is not observed. In a such setup, it would be unwise, for estimating f, to use a conditional approach based upon the probability distribution function of Y given X. Indeed, this approach is affected by the curse of dimensionality, so that the resulting estimator of the residual term E would have considerably a slow rate of convergence if the dimension of …
This paper is a survey of recent results on the adaptive robust non parametric methods for the continuous time regression model with the semi - martingale noises with jumps. The noises are modeled by the Lévy processes, the Ornstein -- Uhlenbeck processes and semi-Markov processes. We represent the general model selection method and the sharp oracle inequalities methods which provide the robust efficient estimation in the adaptive setting. Moreover, we present the recent results on the improved model selection methods for the nonparametric estimation problems.
We propose a time-varying generalization of the Bradley-Terry model that allows for nonparametric modeling of dynamic global rankings of distinct teams. We develop a novel estimator that relies on kernel smoothing to pre-process the pairwise comparisons over time and is applicable in sparse settings where the Bradley-Terry may not be fit. We obtain necessary and sufficient conditions for the existence and uniqueness of our estimator. We also derive time-varying oracle bounds for both the estimation error and the excess risk in the model-agnostic setting where the Bradley-Terry model is not nec…
