The Nadaraya-Watson kernel estimator is among the most popular nonparameteric regression technique thanks to its simplicity. Its asymptotic bias has been studied by Rosenblatt in 1969 and has been reported in a number of related literature. However, Rosenblatt's analysis is only valid for infinitesimal bandwidth. In contrast, we propose in this paper an upper bound of the bias which holds for finite bandwidths. Moreover, contrarily to the classic analysis we allow for discontinuous first order derivative of the regression function, we extend our bounds for multidimensional domains and we include the knowledge of the bound of the regression function when it exists and if it is known, to obtain a tighter bound. We believe that this work has potential applications in those fields where some hard guarantees on the error are needed
We investigate the asymptotic behavior of the Nadaraya-Watson estimator for the estimation of the regression function in a semiparametric regression model. On the one hand, we make use of the recursive version of the sliced inverse regression method for the estimation of the unknown parameter of the model. On the other hand, we implement a recursive Nadaraya-Watson procedure for the estimation of the regression function which takes into account the previous estimation of the parameter of the semiparametric regression model. We establish the almost sure convergence as well as the asymptotic nor…
We study the Nadaraya-Watson (N-W) estimator for the drift function of two-sided reflected stochastic processes. We propose a discrete-type N-W estimator and a continuous-type N-W estimator based on the discretely observed processes and continuously observed processes respectively. Under some regular conditions, we obtain the consistency and give the asymptotic distributions for the two estimators. Furthermore, we briefly remark that our method can be applied to the one-sided reflected stochastic processes spontaneously. Numerical studies show that the proposed estimators is adequate for pract…
In a regression model, we write the Nadaraya-Watson estimator of the regression function as the quotient of two kernel estimators, and propose a bandwidth selection method for both the numerator and the denominator. We prove risk bounds for both data driven estimators and for the resulting ratio. The simulation study confirms that both estimators have good performances, compared to the ones obtained by cross-validation selection of the bandwidth. However, unexpectedly, the single-bandwidth cross-validation estimator is found to be much better than the ratio of the previous two good estimators,…
The celebrated Nadaraya-Watson kernel estimator is among the most studied method for nonparametric regression. A classical result is that its rate of convergence depends on the number of covariates and deteriorates quickly as the dimension grows, which underscores the "curse of dimensionality" and has limited its use in high dimensional settings. In this article, we show that when the true regression function is single or multi-index, the effects of the curse of dimensionality may be mitigated for the Nadaraya-Watson kernel estimator. Specifically, we prove that with $K$-fold cross-validation,…
We consider a class of nonparametric time series regression models in which the regressor takes values in a sequence space. Technical challenges that hampered theoretical advances in these models include the lack of associated Lebesgue density and difficulties with regard to the choice of dependence structure in the autoregressive framework. We propose an infinite-dimensional Nadaraya-Watson type estimator, and investigate its asymptotic properties in detail under both static regressive and autoregressive contexts, aiming to answer the open questions left by Linton and Sancetta (2009). First w…
In this paper, we prove large deviations principle for the Nadaraya-Watson estimator and for the semi-recursive kernel estimator of the regression in the multidimensional case. Under suitable conditions, we show that the rate function is a good rate function. We thus generalize the results already obtained in the unidimensional case for the Nadaraya-Watson estimator. Moreover, we give a moderate deviations principle for these two estimators. It turns out that the rate function obtained in the moderate deviations principle for the semi-recursive estimator is larger than the one obtained for the…
Peter the Great St.Petersburg Polytechnic University（彼得大帝圣彼得堡理工大学）
A new method for estimating the conditional average treatment effect is proposed in the paper. It is called TNW-CATE (the Trainable Nadaraya-Watson regression for CATE) and based on the assumption that the number of controls is rather large whereas the number of treatments is small. TNW-CATE uses the Nadaraya-Watson regression for predicting outcomes of patients from the control and treatment groups. The main idea behind TNW-CATE is to train kernels of the Nadaraya-Watson regression by using a weight sharing neural network of a specific form. The network is trained on controls, and it replaces…
In a pioneer work, Révész (1973) introduces the stochastic approximation method to build up a recursive kernel estimator of the regression function $x\mapsto E(Y|X=x)$. However, according to Révész (1977), his estimator has two main drawbacks: on the one hand, its convergence rate is smaller than that of the nonrecursive Nadaraya-Watson's kernel regression estimator, and, on the other hand, the required assumptions on the density of the random variable $X$ are stronger than those usually needed in the framework of regression estimation. We first come back on the study of the convergence rate o…
We discuss nonparametric tests for parametric specifications of regression quantiles. The test is based on the comparison of parametric and nonparametric fits of these quantiles. The nonparametric fit is a Nadaraya-Watson quantile smoothing estimator. An asymptotic treatment of the test statistic requires the development of new mathematical arguments. An approach that makes only use of plugging in a Bahadur expansion of the nonparametric estimator is not satisfactory. It requires too strong conditions on the dimension and the choice of the bandwidth. Our alternative mathematical approach requi…
In this paper we prove large deviations principles for the averaged stochastic approximation method for the estimation of a regression function introduced by A. Mokkadem et al. [Revisiting Révész's stochastic approximation method for the estimation of a regression function, ALEA Lat. Amm. J. Probab. Math. Stat. 6 (2009), 63-114]. We show that the averaged stochastic approximation algorithm constructed using the weight sequence which minimize the asymptotic variance gives the same pointwise LDP as the Nadaraya-Watson kernel estimator. Moreover, we give a moderate deviations principle for these …
This paper provides the theory about the convergence rate of the tilted version of linear smoother. We study tilted linear smoother, a nonparametric regression function estimator, which is obtained by minimizing the distance to an infinite order flat-top trapezoidal kernel estimator. We prove that the proposed estimator achieves a high level of accuracy. Moreover, it preserves the attractive properties of the infinite order flat-top kernel estimator. We also present an extensive numerical study for analysing the performance of two members of the tilted linear smoother class named tilted Nadara…
This paper is devoted to the parametric estimation of a shift together with the nonparametric estimation of a regression function in a semiparametric regression model. We implement a very efficient and easy to handle Robbins-Monro procedure. On the one hand, we propose a stochastic algorithm similar to that of Robbins-Monro in order to estimate the shift parameter. A preliminary evaluation of the regression function is not necessary to estimate the shift parameter. On the other hand, we make use of a recursive Nadaraya-Watson estimator for the estimation of the regression function. This kernel…
We study the nonparametric Nadaraya-Watson estimator of the drift function for ergodic stochastic processes driven by fractional Brownian motion of Hurst parameter H > 1/2. The estimator is based on the discretely observed stochastic processes. By using the ergodic properties and stochastic integral, we obtain the consistency of the proposed estimator.
The most direct way to express arbitrary dependencies in datasets is to estimate the joint distribution and to apply afterwards the argmax-function to obtain the mode of the corresponding conditional distribution. This method is in practice difficult, because it requires a global optimization of a complicated function, the joint distribution by fixed input variables. This article proposes a method for finding global maxima if the joint distribution is modeled by a kernel density estimation. Some experiments show advantages and shortcomings of the resulting regression method in comparison to th…
We propose a wavelet-based technique for the nonparametric estimation of functions contaminated with noise whose mean and variance are linked via a possibly unknown variance function. Our method, termed the data-driven wavelet-Fisz technique, consists of estimating the variance function via a Nadaraya-Watson estimator, and then performing a wavelet thresholding procedure which uses the estimated variance function and local means of the data to set the thresholds at a suitable level. We demonstrate the mean-square near-optimality of our wavelet estimator over the usual range of Besov classes. T…
We study the local linear estimator for the drift coefficient of stochastic differential equations driven by $α$-stable Lévy motions observed at discrete instants letting $T \rightarrow \infty$. Under regular conditions, we derive the weak consistency and central limit theorem of the estimator. Compare with Nadaraya-Watson estimator, the local linear estimator has a bias reduction whether kernel function is symmetric or not under different schemes.
In this study, we develop an asymptotic theory of nonparametric regression for a locally stationary functional time series. First, we introduce the notion of a locally stationary functional time series (LSFTS) that takes values in a semi-metric space. Then, we propose a nonparametric model for LSFTS with a regression function that changes smoothly over time. We establish the uniform convergence rates of a class of kernel estimators, the Nadaraya-Watson (NW) estimator of the regression function, and a central limit theorem of the NW estimator.
This paper is devoted to the nonparametric estimation of the derivative of the regression function in a nonparametric regression model. We implement a very efficient and easy to handle statistical procedure based on the derivative of the recursive Nadaraya-Watson estimator. We establish the almost sure convergence as well as the asymptotic normality for our estimates. We also illustrate our nonparametric estimation procedure on simulated and real life data associated with sea shores water quality and valvometry.
This article studies a trimmed version of the Nadaraya-Watson estimator to estimate the unknown non-parametric regression function. The characterization of the estimator through minimization problem is established, and its pointwise asymptotic distribution is also derived. The robustness property of the proposed estimator is also studied through breakdown point. Besides, the asymptotic efficiency study along with an extensive simulation study shows that this estimator performs well for various cases. The practicability of the estimator is shown for three benchmark real data as well.
The idea of maximizing the likelihood of the observed range for a set of jointly realized counts has been employed in a variety of contexts. The applicability of the MLE introduced in [1] has been extended to the general case of a multivariate sample containing interval censored outcomes. In addition, a kernel density estimator and a related score function have been proposed leading to the construction of a modified Nadaraya-Watson regression estimator. Finally, the author has treated the problems of estimating the parameters of a mutinomial distribution and the analysis of contingency tables …
In this paper we propose a variable bandwidth kernel regression estimator for $i.i.d.$ observations in $\mathbb{R}^2$ to improve the classical Nadaraya-Watson estimator. The bias is improved to the order of $O(h_n^4)$ under the condition that the fifth order derivative of the density function and the sixth order derivative of the regression function are bounded and continuous. We also establish the central limit theorems for the proposed ideal and true variable kernel regression estimators. The simulation study confirms our results and demonstrates the advantage of the variable bandwidth kerne…
In 1991 Stute introduced a class of estimators called conditional U-statistics. They can be seen as a generalization of the Nadaraya-Watson estimator, and their strong pointwise consistency to the general regression function has been obtained in the same paper by Stute. Very recently, Gine and Mason introduced the notion of a local U-process, which generalizes that of a local empirical process, and obtained central limit theorems and laws of the iterated logarithm for this class. We apply the methods developed by Einmahl and Mason (2005) and Gine and Mason (2007a,b) to establish uniform in ban…
We consider a recurrent Markov process which is an Itô semi-martingale. The Lévy kernel describes the law of its jumps. Based on observations X(0),X(Δ),...,X(nΔ), we construct an estimator for the Lévy kernel's density. We prove its consistency (as nΔ->\infty and Δ->0) and a central limit theorem. In the positive recurrent case, our estimator is asymptotically normal; in the null recurrent case, it is asymptotically mixed normal. Our estimator's rate of convergence equals the non-parametric minimax rate of smooth density estimation. The asymptotic bias and variance are analogous to those of th…
Let X be a d dimensional vector of covariates and Y be the response variable. Under the nonparametric model Y = m(X) + σ(X) \in we develop an ANOVA-type test for the null hypothesis that a particular coordinate of X has no influence on the regression function. The asymptotic distribution of the test statistic, using residuals based on Nadaraya-Watson type kernel estimator and d \leq 4, is established under the null hypothesis and local alternatives. Simulations suggest that under a sparse model, the applicability of the test extends to arbitrary d through sufficient dimension reduction. Using …
We consider the double functional nonparametric regression model $Y=r(X)+ε$, where the response variable $Y$ is Hilbert space-valued and the covariate $X$ takes values in a pseudometric space. The data satisfy an ergodicity criterion which dates back to Laib and Louani (2010) and are arranged in a triangular array. So our model also applies to samples obtained from spatial processes, e.g., stationary random fields indexed by the regular lattice $\mathbb{Z}^N$ for some $N\in\mathbb{N}_+$. We consider a kernel estimator of the Nadaraya--Watson type for the regression operator $r$ and study its l…
In this article, we consider the estimation of the structural change point in the nonparametric model with dependent observations. We introduce a maximum-CUSUM-estimation procedure, where the CUSUM statistic is constructed based on the sum-of-squares aggregation of the difference of the two Nadaraya-Watson estimates using the observations before and after a specific time point. Under some mild conditions, we prove that the statistic tends to zero almost surely if there is no change, and is larger than a threshold asymptotically almost surely otherwise, which helps us to obtain a threshold-dete…
Changes on temperature patterns, on a local scale, are perceived by individuals as the most direct indicators of global warming and climate change. As a specific example, for an Atlantic climate location, spring and fall seasons should present a mild transition between winter and summer, and summer and winter, respectively. By observing daily temperature curves along time, being each curve attached to a certain calendar day, a regression model for these variables (temperature curve as covariate and calendar day as response) would be useful for modeling their relation for a certain period. In a…
Dirichlet kernel density estimation on the simplex with missing data
Imam Mohammad Ibn Saud Islamic University（伊玛目穆罕默德·本·沙特伊斯兰大学）；Université de Tunis El-Manar（突尼斯埃尔曼纳尔大学）；Dalhousie University（达尔豪斯大学）；Université du Québec à Trois-Rivières（魁北克特罗瓦里维尔斯大学）
On the Dirichlet-kernel Gasser--Müller estimator and its competitors for fixed design regression on the simplex
Université de Tunis El-Manar（突尼斯埃尔马纳大学）；McGill University（麦吉尔大学）；Univ Rennes, CNRS, IRMAR - UMR 6625（雷恩大学，法国国家科学研究中心，IRMAR - UMR 6625）；Université du Québec à Trois-Rivières（魁北克省特鲁瓦里维埃尔斯大学）
This paper develops a novel approach to density estimation on a network. We formulate nonparametric density estimation on a network as a nonparametric regression problem by binning. Nonparametric regression using local polynomial kernel-weighted least squares have been studied rigorously, and its asymptotic properties make it superior to kernel estimators such as the Nadaraya-Watson estimator. When applied to a network, the best estimator near a vertex depends on the amount of smoothness at the vertex. Often, there are no compelling reasons to assume that a density will be continuous or discon…
Data loss is a critical problem in structural health monitoring (SHM). Probability distributions play a highly important role in many applications. Improving the quality of distribution estimations made using incomplete samples is highly important. Missing samples can be compensated for by applying conventional missing data restoration methods; however, ensuring that restored samples roughly follow underlying distributions of true missing data remains a challenge. Another strategy involves directly restoring the probability density function (PDF) for a sensor when samples are missing by levera…
Let $v$ be a vector field in a bounded open set $G\subset {\mathbb {R}}^d$. Suppose that $v$ is observed with a random noise at random points $X_i, i=1,...,n,$ that are independent and uniformly distributed in $G.$ The problem is to estimate the integral curve of the differential equation \[\frac{dx(t)}{dt}=v(x(t)),\qquad t\geq 0,x(0)=x_0\in G,\] starting at a given point $x(0)=x_0\in G$ and to develop statistical tests for the hypothesis that the integral curve reaches a specified set $Γ\subset G.$ We develop an estimation procedure based on a Nadaraya--Watson type kernel regression estimator…
Peter the Great St.Petersburg Polytechnic University（彼得大帝圣彼得堡理工大学）
A method for estimating the conditional average treatment effect under condition of censored time-to-event data called BENK (the Beran Estimator with Neural Kernels) is proposed. The main idea behind the method is to apply the Beran estimator for estimating the survival functions of controls and treatments. Instead of typical kernel functions in the Beran estimator, it is proposed to implement kernels in the form of neural networks of a specific form called the neural kernels. The conditional average treatment effect is estimated by using the survival functions as outcomes of the control and t…
