We consider the nonparametric robust estimation problem for regression models in continuous time with semi-Markov noises. An adaptive model selection procedure is proposed. Under general moment conditions on the noise distribution a sharp non-asymptotic oracle inequality for the robust risks is obtained and the robust efficiency is shown. It turns out that for semi-Markov models the robust minimax convergence rate may be faster or slower than the classical one.
Three types of regression models researchers need to be familiar with and know the requirements of each: parametric, semiparametric and nonparametric regression models. The type of modeling used is based on how much information are available about the form of the relationship between response variable and explanatory variables, and the random error distribution. In this article, differences between models, common methods of estimation, robust estimation, and applications are introduced. The R code for all the graphs and analyses presented here, in this article, is available in the Appendix.
It is proved that nonparametric autoregression is asymptotically equivalent in the sense of Le Cam's deficiency distance to nonparametric regression with random design as well as with regular nonrandom design.
An adaptive nonparametric estimation procedure is constructed for heteroscedastic regression when the noise variance depends on the unknown regression. A non-asymptotic upper bound for a quadratic risk (oracle inequality) is obtained
An adaptive nonparametric estimation procedure is constructed for the estimation problem of heteroscedastic regression when the noise variance depends on the unknown regression. A non-asymptotic upper bound for a quadratic risk (an oracle inequality) is constructed.
This manuscript is a supplemental document providing the omitted material for our paper entitled "Nonparametric kernel estimation of the probability density function of regression errors using estimated residuals" [arXiv:1010.0439]. The paper is submitted to Journal of Nonparametric Statistics.
This note shows that adding monotonicity or convexity constraints on the regression function does not restore well-posedness in nonparametric instrumental variable regression. The minimum distance problem without regularisation is still locally ill-posed.
In this paper, we develop nonparametric inference on spatial regression models as an extension of Lu and Tj\ostheim(2014), which develops nonparametric inference on density functions of stationary spatial processes under domain expanding and infill (DEI) asymptotics. In particular, we derive multivariate central limit theorems of mean and variance functions of nonparametric spatial regression models. Built upon those results, we propose a method to construct confidence bands for mean and variance functions.
We construct an efficient estimator for the error distribution function of the nonparametric regression model Y = r(Z) + e. Our estimator is a kernel smoothed empirical distribution function based on residuals from an under-smoothed local quadratic smoother for the regression function.
This paper concerns the estimation of the regression function at a given point in nonparametric heteroscedastic models with Gaussian noise or with noise having unknown distribution. In the two cases an asymptotically efficient kernel estimator is constructed for the minimax absolute error risk.
In this paper, we are concerned with nonparametric estimation of the multivariate regression function in the presence of right censored data. More precisely, we propose a statistic that is shown to be asymptotically normally distributed under the additive assumption, and that could be used to test for additivity in the censored regression setting.
We establish the asymptotic normality of the kernel type estimator for the regression function constructed from quasi-associated data when the explanatory variable takes its values in a separable Hilbert space.
This paper is devoted to the nonparametric estimation of the derivative of the regression function in a nonparametric regression model. We implement a very efficient and easy to handle statistical procedure based on the derivative of the recursive Nadaraya-Watson estimator. We establish the almost sure convergence as well as the asymptotic normality for our estimates. We also illustrate our nonparametric estimation procedure on simulated and real life data associated with sea shores water quality and valvometry.
The detection of influential observations for the standard least squares regression model is a question that has been extensively studied. LAD regression diagnostics offers alternative approaches whose main feature is the robustness. In this paper a new approach for nonparametric detection of influencial observations in LAD regression models is presented and compared with other classical methods of diagnostics.
In this study, we develop an asymptotic theory of nonparametric regression for a locally stationary functional time series. First, we introduce the notion of a locally stationary functional time series (LSFTS) that takes values in a semi-metric space. Then, we propose a nonparametric model for LSFTS with a regression function that changes smoothly over time. We establish the uniform convergence rates of a class of kernel estimators, the Nadaraya-Watson (NW) estimator of the regression function, and a central limit theorem of the NW estimator.
We introduce and study a local linear nonparametric regression estimator for censorship model. The main goal of this paper is, to establish the uniform almost sure consistency result with rate over a compact set for the new estimate. To support our theoretical result, a simulation study has been done to make comparison with the classical regression estimator.
This paper discusses a design-dependent nature of variance in nonparametric link regression aiming at predicting a mean outcome at a link, i.e., a pair of nodes, based on currently observed data comprising covariates at nodes and outcomes at links.
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper (2007) for estimation of unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk. It means that the asymptotic quadratic risk for this procedure coincides with a sharp lower bound.
We analyze the statistical properties of nonparametric regression estimators using covariates which are not directly observable, but have be estimated from data in a preliminary step. These so-called generated covariates appear in numerous applications, including two-stage nonparametric regression, estimation of simultaneous equation models or censored regression models. Yet so far there seems to be no general theory for their impact on the final estimator's statistical properties. Our paper provides such results. We derive a stochastic expansion that characterizes the influence of the generat…
We show that nonparametric regression is asymptotically equivalent in Le Cam's sense with a sequence of Gaussian white noise experiments as the number of observations tends to infinity. We propose a general constructive framework based on approximation spaces, which permits to achieve asymptotic equivalence even in the cases of multivariate and random design.
In this paper we prove large deviations principles for the Nadaraya-Watson estimator of the regression of a real-valued variable with a functional covariate. Under suitable conditions, we show pointwise and uniform large deviations theorems with good rate functions.
We discuss nonparametric tests for parametric specifications of regression quantiles. The test is based on the comparison of parametric and nonparametric fits of these quantiles. The nonparametric fit is a Nadaraya-Watson quantile smoothing estimator. An asymptotic treatment of the test statistic requires the development of new mathematical arguments. An approach that makes only use of plugging in a Bahadur expansion of the nonparametric estimator is not satisfactory. It requires too strong conditions on the dimension and the choice of the bandwidth. Our alternative mathematical approach requi…
A nonparametric regression setting is considered with a real-valued covariate and responses from a metric space. One may approach this setting via Fréchet regression, where the value of the regression function at each point is estimated via a Fréchet mean calculated from an estimated objective function. A second approach is geodesic regression, which builds upon fitting geodesics to observations by a least squares method. These approaches are applied to transform two of the most important nonparametric regression estimators in statistics to the metric setting -- the local linear regression est…
Suppose the nonparametric regression function of a response variable $Y$ on covariates $X$ and $Z$ is an affine function of $X$ such that the slope $β$ and the intercept $α$ are real valued measurable functions on the range of the completely arbitrary random element $Z$. Assume that $X$ has a finite moment of order greater than or equal to $2$, $Y$ has a finite moment of conjugate order, and $α\left(Z\right)$ and $α\left(Z\right)X$ have finite first moments. Then, the nonparametric regression function equals the least squares linear regression function of $Y$ on $X$ with all the moments that a…
We consider nonparametric estimation for functional autoregressive processes with Markov switching. First, we study the case where complete data is available; i.e. when we observe the Markov switching regime. Then we estimate the regression function in each regime using a Nadaraya-Watson type estimator. Second, we introduce a nonparametric recursive algorithm in the case of hidden Markov switching regime. Our algorithm restores the missing data by means of a Monte-Carlo step and estimate the regression function via a Robbins-Monro step. Consistency of the estimators are proved in both cases. F…
Nonparametric identification and maximum likelihood estimation for finite-state hidden Markov models are investigated. We obtain identification of the parameters as well as the order of the Markov chain if the transition probability matrices have full-rank and are ergodic, and if the state-dependent distributions are all distinct, but not necessarily linearly independent. Based on this identification result, we develop nonparametric maximum likelihood estimation theory. First, we show that the asymptotic contrast, the Kullback--Leibler divergence of the hidden Markov model, identifies the true…
In this note, we consider the problem of existence of adaptive confidence bands in the fixed design regression model, adapting ideas in Hoffmann and Nickl (2011) to the present case. In the course of the proof, we show that sup-norm adaptive estimators exist as well in regression.
In the context of nonparametric regression, we study conditions under which the consistency (and rates of convergence) of estimators built from discretely sampled curves can be derived from the consistency of estimators based on the unobserved whole trajectories. As a consequence, we derive asymptotic results for most of the regularization techniques used in functional data analysis, including smoothing and basis representation.
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating a unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimators.
In this paper, we built a new nonparametric regression estimator with the local linear method by using the mean squared relative error as a loss function when the data are subject to random right censoring. We establish the uniform almost sure consistency with rate over a compact set of the proposed estimator. Some simulations are given to show the asymptotic behavior of the estimate in different cases.
This paper deals with the nonparametric density estimation of the regression error term assuming its independence with the covariate. The difference between the feasible estimator which uses the estimated residuals and the unfeasible one using the true residuals is studied. An optimal choice of the bandwidth used to estimate the residuals is given. We also study the asymptotic normality of the feasible kernel estimator and its rate-optimality.
This paper develops a method to construct uniform confidence bands for a nonparametric regression function where a predictor variable is subject to a measurement error. We allow for the distribution of the measurement error to be unknown, but assume the availability of validation data or repeated measurements on the latent predictor variable. The proposed confidence band builds on the deconvolution kernel estimation and a novel application of the multiplier bootstrap method. We establish asymptotic validity of the proposed confidence band. To our knowledge, this is the first paper to derive as…
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating an unknown nonparametric regression. %\cite{GaPe1}. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimates
We develop a unifying framework for Bayesian nonparametric regression to study the rates of contraction with respect to the integrated $L_2$-distance without assuming the regression function space to be uniformly bounded. The framework is very flexible and can be applied to a wide class of nonparametric prior models. Three non-trivial applications of the proposed framework are provided: The finite random series regression of an $α$-Hölder function, with adaptive rates of contraction up to a logarithmic factor; The un-modified block prior regression of an $α$-Sobolev function, with adaptive-and…
The main purpose is to estimate the regression function of a real random variable with functional explanatory variable by using a recursive nonparametric kernel approach. The mean square error and the almost sure convergence of a family of recursive kernel estimates of the regression function are derived. These results are established with rates and precise evaluation of the constant terms. Also, a central limit theorem for this class of estimators is established. The method is evaluated on simulations and real data set studies.
In this work, we develop and study an empirical projection operator scheme for solving nonparametric regression problems. This scheme is based on an approximate projection of the regression function over a suitable reproducing kernel Hilbert space (RKHS). The RKHS considered in this paper are generated by the Mercer kernels given by the Legendre Christoffel-Darboux and convolution Sinc kernels. We provide error and convergence analysis of the proposed scheme under the assumption that the regression function belongs to some suitable functional spaces. We also consider the popular RKHS regulariz…
Measure-valued Markov chains have raised interest in Bayesian nonparametrics since the seminal paper by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579--585) where a Markov chain having the law of the Dirichlet process as unique invariant measure has been introduced. In the present paper, we propose and investigate a new class of measure-valued Markov chains defined via exchangeable sequences of random variables. Asymptotic properties for this new class are derived and applications related to Bayesian nonparametric mixture modeling, and to a generalization of the Markov chain proposed by (M…
Most results in nonparametric regression theory are developed only for the case of additive noise. In such a setting many smoothing techniques including wavelet thresholding methods have been developed and shown to be highly adaptive. In this paper we consider nonparametric regression in exponential families with the main focus on the natural exponential families with a quadratic variance function, which include, for example, Poisson regression, binomial regression and gamma regression. We propose a unified approach of using a mean-matching variance stabilizing transformation to turn the relat…
Principled nonparametric tests for regression curvature in $\mathbb{R}^{d}$ are often statistically and computationally challenging. This paper introduces the stratified incomplete local simplex (SILS) tests for joint concavity of nonparametric multiple regression. The SILS tests with suitable bootstrap calibration are shown to achieve simultaneous guarantees on dimension-free computational complexity, polynomial decay of the uniform error-in-size, and power consistency for general (global and local) alternatives. To establish these results, a general theory for incomplete $U$-processes with s…
In this paper, a lower bound is determined in the minimax sense for change point estimators of the first derivative of a regression function in the fractional white noise model. Similar minimax results presented previously in the area focus on change points in the derivatives of a regression function in the white noise model or consider estimation of the regression function in the presence of correlated errors.
We consider the recursive estimation of a regression functional where the explanatory variables take values in some functional space. We prove the almost sure convergence of such estimates for dependent functional data. Also we derive the mean quadratic error of the considered class of estimators. Our results are established with rates and asymptotic appear bounds, under strong mixing condition.
We consider nonparametric estimation of mean regression and conditional variance (or volatility) functions in nonlinear stochastic regression models. Simultaneous confidence bands are constructed and the coverage probabilities are shown to be asymptotically correct. The imposed dependence structure allows applications in many linear and nonlinear auto-regressive processes. The results are applied to the S&P 500 Index data.
This paper considers hidden Markov models where the observations are given as the sum of a latent state which lies in a general state space and some independent noise with unknown distribution. It is shown that these fully nonparametric translation models are identifiable with respect to both the distribution of the latent variables and the distribution of the noise, under mostly a light tail assumption on the latent variables. Two nonparametric estimation methods are proposed and we prove that the corresponding estimators are consistent for the weak convergence topology. These results are ill…
Asymptotic equivalence in Le Cam's sense for nonparametric regression experiments is extended to the case of non-regular error densities, which have jump discontinuities at their endpoints. We prove asymptotic equivalence of such regression models and the observation of two independent Poisson point processes which contain the target curve as the support boundary of its intensity function. The intensity of the point processes is of order of the sample size $n$ and involves the jump sizes as well as the design density. The statistical model significantly differs from regression problems with Ga…
Finite state space hidden Markov models are flexible tools to model phenomena with complex time dependencies: any process distribution can be approximated by a hidden Markov model with enough hidden states.We consider the problem of estimating an unknown process distribution using nonparametric hidden Markov models in the misspecified setting, that is when the data-generating process may not be a hidden Markov model.We show that when the true distribution is exponentially mixing and satisfies a forgetting assumption, the maximum likelihood estimator recovers the best approximation of the true …
This paper presents a nonparametric method for estimating the conditional density associated to the jump rate of a piecewise-deterministic Markov process. In our framework, the estimation needs only one observation of the process within a long time interval. Our method relies on a generalization of Aalen's multiplicative intensity model. We prove the uniform consistency of our estimator, under some reasonable assumptions related to the primitive characteristics of the process. A simulation example illustrates the behavior of our estimator.
This paper considers a general class of nonparametric time series regression models where the regression function can be time-dependent. We establish an asymptotic theory for estimates of the time-varying regression functions. For this general class of models, an important issue in practice is to address the necessity of modeling the regression function as nonlinear and time-varying. To tackle this, we propose an information criterion and prove its selection consistency property. The results are applied to the U.S. Treasury interest rate data.
This paper deals with the consistency of the least squares estimator of a convex regression function when the predictor is multidimensional. We characterize and discuss the computation of such an estimator via the solution of certain quadratic and linear programs. Mild sufficient conditions for the consistency of this estimator and its subdifferentials in fixed and stochastic design regression settings are provided. We also consider a regression function which is known to be convex and componentwise nonincreasing and discuss the characterization, computation and consistency of its least square…
In this paper we consider the nonparametric estimation of density and regression functions with non-negative support using a gamma kernel procedure introduced by Chen (2000). Strong uniform consistency and asymptotic normality of the corresponding estimators are established under a general ergodic assumption on the data generation process. Our results generalize those of Shi and Song (2016), obtained in the classic i.i.d. framework, and the works of Bouezmarni and Rombouts (2008, 2010b) and Gospodinov and Hirukawa (2007) for mixing time series.
Efficient estimation under bias sampling, censoring or truncation is a difficult question which has been partially answered and the usual estimators are not always consistent. Several biased designs are considered for models with variables $(X,Y)$ where $Y$ is an indicator and $X$ an explanatory variable, or for continuous variables $(X,Y)$. The identifiability of the models are discussed. New nonparametric estimators of the regression functions and conditional quantiles are proposed.
Asymptotic equivalence results for nonparametric regression experiments have always assumed that the variances of the observations are known. In practice, however the variance of each observation is generally considered to be an unknown nuisance parameter. We establish an asymptotic approximation to the nonparametric regression experiment when the value of the variance is an additional parameter to be estimated or tested. This asymptotically equivalent experiment has two components: the first contains all the information about the variance and the second has all the information about the mean.…
In this paper we propose a new test for the hypothesis of a constant coefficient of variation in the common nonparametric regression model. The test is based on an estimate of the $L^2$-distance between the square of the regression function and variance function. We prove asymptotic normality of a standardized estimate of this distance under the null hypothesis and fixed alternatives and the finite sample properties of a corresponding bootstrap test are investigated by means of a simulation study. The results are applicable to stationary processes with the common mixing conditions and are used…
We propose a new, two-step empirical Bayes-type of approach for neural networks. We show in context of the nonparametric regression model that the procedure (up to a logarithmic factor) provides optimal recovery of the underlying functional parameter of interest and provides Bayesian credible sets with frequentist coverage guarantees. The approach requires fitting the neural network only once, hence it is substantially faster than Bootstrapping type approaches. We demonstrate the applicability of our method over synthetic data, observing good estimation properties and reliable uncertainty quan…
We construct nonparametric confidence sets for regression functions using wavelets that are uniform over Besov balls. We consider both thresholding and modulation estimators for the wavelet coefficients. The confidence set is obtained by showing that a pivot process, constructed from the loss function, converges uniformly to a mean zero Gaussian process. Inverting this pivot yields a confidence set for the wavelet coefficients, and from this we obtain confidence sets on functionals of the regression curve.
