Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regression
Xiaohong Chen, Timothy M. Christensen
备注：
This paper is a major extension of Sections 2 and 3 of our Cowles Foundation Discussion Paper CFDP1923, Cemmap Working Paper CWP56/13 and arXiv preprint arXiv:1311.0412 [math.ST]. Section 3 of the previous version of this paper (dealing with data-driven choice of sieve dimension) is currently being revised as a separate paper
This paper makes several important contributions to the literature about nonparametric instrumental variables (NPIV) estimation and inference on a structural function $h_0$ and its functionals. First, we derive sup-norm convergence rates for computationally simple sieve NPIV (series 2SLS) estimators of $h_0$ and its derivatives. Second, we derive a lower bound that describes the best possible (minimax) sup-norm rates of estimating $h_0$ and its derivatives, and show that the sieve NPIV estimator can attain the minimax rates when $h_0$ is approximated via a spline or wavelet sieve. Our optimal sup-norm rates surprisingly coincide with the optimal root-mean-squared rates for severely ill-posed problems, and are only a logarithmic factor slower than the optimal root-mean-squared rates for mildly ill-posed problems. Third, we use our sup-norm rates to establish the uniform Gaussian process strong approximations and the score bootstrap uniform confidence bands (UCBs) for collections of nonlinear functionals of $h_0$ under primitive conditions, allowing for mildly and severely ill-posed problems. Fourth, as applications, we obtain the first asymptotic pointwise and uniform inference results for plug-in sieve t-statistics of exact consumer surplus (CS) and deadweight loss (DL) welfare functionals under low-level conditions when demand is estimated via sieve NPIV. Empiricists could read our real data application of UCBs for exact CS and DL functionals of gasoline demand that reveals interesting patterns and is applicable to other markets.
This paper establishes a necessary and sufficient condition for the asymptotic normality of the nonparametric estimator of sample coverage proposed by Good [Biometrica 40 (1953) 237--264]. This new necessary and sufficient condition extends the validity of the asymptotic normality beyond the previously proven cases.
We derive asymptotic normality of kernel type deconvolution density estimators. In particular we consider deconvolution problems where the known component of the convolution has a symmetric lambda-stable distribution, 0<lambda<= 2. It turns out that the limit behavior changes if the exponent parameter lambda passes the value one, the case of Cauchy deconvolution.
We propose nonparametric estimators for conditional value-at-risk (CVaR) and conditional expected shortfall (CES) associated with conditional distributions of a series of returns on a financial asset. The return series and the conditioning covariates, which may include lagged returns and other exogenous variables, are assumed to be strong mixing and follow a nonparametric conditional location-scale model. First stage nonparametric estimators for location and scale are combined with a generalized Pareto approximation for distribution tails proposed by Pickands (1975) to give final estimators fo…
The Highly-Adaptive-LASSO Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functional parameters (i.e., the relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuisance functional parameters by minimizing the empirical risk over the parameter space under the constraint that the sectional variation norm of the candidate functions are bounded by a constant, where …
We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilks's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression involving functional data. Our numerical results demonstrated the improved performance of empirical likelihood over approximation based on asymptotic normality.
We establish the asymptotic normality of the kernel type estimator for the regression function constructed from quasi-associated data when the explanatory variable takes its values in a separable Hilbert space.
Cowles Foundation for Research in Economics, Yale University（耶鲁大学考尔斯经济学研究基金会）；George Washington University（乔治华盛顿大学）
We study the off-policy evaluation (OPE) problem in an infinite-horizon Markov decision process with continuous states and actions. We recast the $Q$-function estimation into a special form of the nonparametric instrumental variables (NPIV) estimation problem. We first show that under one mild condition the NPIV formulation of $Q$-function estimation is well-posed in the sense of $L^2$-measure of ill-posedness with respect to the data generating distribution, bypassing a strong assumption on the discount factor $γ$ imposed in the recent literature for obtaining the $L^2$ convergence rates of v…
The accuracy of a diagnostic test is typically characterised using the receiver operating characteristic (ROC) curve. Summarising indexes such as the area under the ROC curve (AUC) are used to compare different tests as well as to measure the difference between two populations. Often additional information is available on some of the covariates which are known to influence the accuracy of such measures. We propose nonparametric methods for covariate adjustment of the AUC. Models with normal errors and non-normal errors are discussed and analysed separately. Nonparametric regression is used for…
We consider inference procedures, conditional on an observed ancillary statistic, for regression coefficients under a linear regression setup where the unknown error distribution is specified nonparametrically. We establish conditional asymptotic normality of the regression coefficient estimators under regularity conditions, and formally justify the approach of plugging in kernel-type density estimators in conditional inference procedures. Simulation results show that the approach yields accurate conditional coverage probabilities when used for constructing confidence intervals. The plug-in ap…
A compound Poisson process whose parameters are all unknown is observed at finitely many equispaced times. Nonparametric estimators of the jump and Lévy distributions are proposed and functional central limit theorems using the uniform norm are proved for both under mild conditions. The limiting Gaussian processes are identified and efficiency of the estimators is established. Kernel estimators for the mass function, the intensity and the drift are also proposed, their asymptotic properties including efficiency are analysed, and joint asymptotic normality is shown. Inference tools such as conf…
We study the nonparametric estimators of the infinitesimal coefficients of the second-order jump-diffusion models. Under the mild conditions, we obtain the weak consistency and the asymptotic normalities of the estimators.
In this paper, we are concerned with nonparametric estimation of the multivariate regression function in the presence of right censored data. More precisely, we propose a statistic that is shown to be asymptotically normally distributed under the additive assumption, and that could be used to test for additivity in the censored regression setting.
Hypothesis tests are a crucial statistical tool for data mining and are the workhorse of scientific research in many fields. Here we study differentially private tests of independence between a categorical and a continuous variable. We take as our starting point traditional nonparametric tests, which require no distributional assumption (e.g., normality) about the data distribution. We present private analogues of the Kruskal-Wallis, Mann-Whitney, and Wilcoxon signed-rank tests, as well as the parametric one-sample t-test. These tests use novel test statistics developed specifically for the pr…
We study the nonparametric estimation for the intensity of Poisson random measure in jump-diffusion CIR model based on the low frequency observations. This is given in terms of the minimization of norms on a nonempty, closed and convex subset of some special Hilbert space. We establish the measurability of the estimator and derive its consistency and asymptotic risk bound.
We consider problem of signal detection in Gaussian white noise. Test statistics are linear combinations of squares of estimators of Fourier coefficients or $\mathbb{L}_2$-norms of kernel estimators. We point out necessary and sufficient conditions when nonparametric sets of alternatives have a given rate of exponential decay for type II error probabilities.
In this paper we examine the use of topological methods for multivariate statistics. Using persistent homology from computational algebraic topology, a random sample is used to construct estimators of persistent homology. This estimation procedure can then be evaluated using the bottleneck distance between the estimated persistent homology and the true persistent homology. The connection to statistics comes from the fact that when viewed as a nonparametric regression problem, the bottleneck distance is bounded by the sup-norm loss. Consequently, a sharp asymptotic minimax bound is determined u…
We present a nonparametric family of estimators for the tail index of a Pareto-type distribution when covariate information is available. Our estimators are based on a weighted sum of the log-spacings between some selected observations. This selection is achieved through a moving window approach on the covariate domain and a random threshold on the variable of interest. Asymptotic normality is proved under mild regularity conditions and illustrated for some weight functions. Finite sample performances are presented on a real data study.
Verification bias is a well known problem when the predictive ability of a diagnostic test has to be evaluated. In this paper, we discuss how to assess the accuracy of continuous-scale diagnostic tests in the presence of verification bias, when a three-class disease status is considered. In particular, we propose a fully nonparametric verification bias-corrected estimator of the ROC surface. Our approach is based on nearest-neighbor imputation and adopts generic smooth regression models for both the disease and the verification processes. Consistency and asymptotic normality of the proposed es…
This paper is devoted to the nonparametric estimation of the derivative of the regression function in a nonparametric regression model. We implement a very efficient and easy to handle statistical procedure based on the derivative of the recursive Nadaraya-Watson estimator. We establish the almost sure convergence as well as the asymptotic normality for our estimates. We also illustrate our nonparametric estimation procedure on simulated and real life data associated with sea shores water quality and valvometry.
Confidence intervals and joint confidence sets are constructed for the nonparametric calibration of exponential Lévy models based on prices of European options. To this end, we show joint asymptotic normality in the spectral calibration method for the estimators of the volatility, the drift, the jump intensity and the Lévy density at finitely many points.
In this paper, we consider the filtering and smoothing recursions in nonparametric finite state space hidden Markov models (HMMs) when the parameters of the model are unknown and replaced by estimators. We provide an explicit and time uniform control of the filtering and smoothing errors in total variation norm as a function of the parameter estimation errors. We prove that the risk for the filtering and smoothing errors may be uniformly upper bounded by the risk of the estimators. It has been proved very recently that statistical inference for finite state space nonparametric HMMs is possible…
Practical applications of nonparametric density estimators in more than three dimensions suffer a great deal from the well-known curse of dimensionality: convergence slows down as dimension increases. We show that one can evade the curse of dimensionality by assuming a simplified vine copula model for the dependence between variables. We formulate a general nonparametric estimator for such a model and show under high-level assumptions that the speed of convergence is independent of dimension. We further discuss a particular implementation for which we validate the high-level assumptions and es…
Let X be a second order random process indexed by a compact interval [0,T]. Assume that n independent realizations of X are observed on a fixed grid of p time points. Under mild regularity assumptions on the sample paths of X, we show the asymptotic normality of suitable nonparametric estimators of the trend function mu = EX in the space C([0,T]) as n, p go to infinity and, using Gaussian process theory, we derive approximate simultaneous confidence bands for mu.
We consider the problem of nonparametric estimation of the drift of a continuously observed one-dimensional diffusion with periodic drift. Motivated by computational considerations, van der Meulen e.a. (2014) defined a prior on the drift as a randomly truncated and randomly scaled Faber-Schauder series expansion with Gaussian coefficients. We study the behaviour of the posterior obtained from the prior from a frequentist asymptotic point of view. If the true data generating drift is smooth, it is proved that the posterior is adaptive with posterior contraction rates for the $L_2$-norm that are…
This paper deals with the nonparametric density estimation of the regression error term assuming its independence with the covariate. The difference between the feasible estimator which uses the estimated residuals and the unfeasible one using the true residuals is studied. An optimal choice of the bandwidth used to estimate the residuals is given. We also study the asymptotic normality of the feasible kernel estimator and its rate-optimality.
We point out necessary and sufficient conditions of uniform consistency of nonparametric sets of alternatives for widespread nonparametric tests. Nonparametric sets of alternatives can be defined both in terms of distribution function and in terms of density (or signals in the problem of signal detection in Gaussian white noise). In this part of paper such conditions are provided for $χ^2-$tests with increasing number of cells, Cramer-von Mises tests, tests generated $\mathbb{L}_2$- norms of kernel estimators and tests generated quadratic forms of estimators of Fourier coefficients.
Using and extending fractional order statistic theory, we characterize the $O(n^{-1})$ coverage probability error of the previously proposed confidence intervals for population quantiles using $L$-statistics as endpoints in Hutson (1999). We derive an analytic expression for the $n^{-1}$ term, which may be used to calibrate the nominal coverage level to get $O\bigl(n^{-3/2}[\log(n)]^3\bigr)$ coverage error. Asymptotic power is shown to be optimal. Using kernel smoothing, we propose a related method for nonparametric inference on conditional quantiles. This new method compares favorably with as…
This paper describes an estimator of the additive components of a nonparametric additive model with a known link function. When the additive components are twice continuously differentiable, the estimator is asymptotically normally distributed with a rate of convergence in probability of n^{-2/5}. This is true regardless of the (finite) dimension of the explanatory variable. Thus, in contrast to the existing asymptotically normal estimator, the new estimator has no curse of dimensionality. Moreover, the estimator has an oracle property. The asymptotic distribution of each additive component is…
In this study, we develop an asymptotic theory of nonparametric regression for locally stationary random fields (LSRFs) $\{{\bf X}_{{\bf s}, A_{n}}: {\bf s} \in R_{n} \}$ in $\mathbb{R}^{p}$ observed at irregularly spaced locations in $R_{n} =[0,A_{n}]^{d} \subset \mathbb{R}^{d}$. We first derive the uniform convergence rate of general kernel estimators, followed by the asymptotic normality of an estimator for the mean function of the model. Moreover, we consider additive models to avoid the curse of dimensionality arising from the dependence of the convergence rate of estimators on the number…
Nonparametric Correlation Estimator via Solving Cubic Equations and its Application to Brain Functional Connectivity Analysis
Lijiang Culture and Tourism College（丽江文化旅游学院）；York Neuroimaging Centre, University of York（约克大学神经影像中心）；School of Mathematics, Statistics and Actuarial Science, University of Kent（肯特大学数学、统计与精算科学学院）
We discuss nonparametric estimators of the distribution of the incubation time of a disease. The classical approach in these models is to use parametric families like Weibull, log-normal or gamma in the estimation procedure. We analyze instead the nonparametric maximum likelihood estimator (MLE) and show that, under some conditions, its rate of convergence is cube root $n$ and that its limit behavior is given by Chernoff's distribution. We also study smooth estimates, based on the MLE. The density estimates, based on the MLE, are capable of catching finer or unexpected aspects of the density, …
For $χ^2-$tests with increasing number of cells, Cramer-von Mises tests, tests generated $\mathbb{L}_2$- norms of kernel estimators and tests generated quadratic forms of estimators of Fourier coefficients, we find necessary and sufficient conditions of consistency and inconsistency for sequences of alternatives having a given rate of convergence to hypothesis in $\mathbb{L}_2$-norm. We provide transparent interpretations of these conditions allowing to understand the structure of such consistent sequences. For problem of signal detection in Gaussian white noise we show that, if set of alterna…
With regard to a three-step estimation procedure, proposed without theoretical discussion by Li and You in Journal of Applied Statistics and Management, for a nonparametric regression model with time-varying regression function, local stationary regressors and time-varying AR(p) (tvAR(p)) error process , we established all necessary asymptotic properties for each of estimator. We derive the convergence rate and asymptotic normality of the preliminary estimation of nonparametric regression function, establish the asymptotic distribution of time-varying coefficient functions in the error term, a…
Estimation of genewise variance arises from two important applications in microarray data analysis: selecting significantly differentially expressed genes and validation tests for normalization of microarray data. We approach the problem by introducing a two-way nonparametric model, which is an extension of the famous Neyman--Scott model and is applicable beyond microarray data. The problem itself poses interesting challenges because the number of nuisance parameters is proportional to the sample size and it is not obvious how the variance function can be estimated when measurements are correl…
We consider nonparametric regression in the context of functional data, that is, when a random sample of functions is observed on a fine grid. We obtain a functional asymptotic normality result allowing to build simultaneous confidence bands (SCB) for various estimation and inference tasks. Two applications to a SCB procedure for the regression function and to a goodness-of-fit test for curvilinear regression models are proposed. The first one has improved accuracy upon the other available methods while the second can detect local departures from a parametric shape, as opposed to the usual goo…
Consider a Poisson point process with unknown support boundary curve $g$, which forms a prototype of an irregular statistical model. We address the problem of estimating non-linear functionals of the form $\int Φ(g(x))\,dx$. Following a nonparametric maximum-likelihood approach, we construct an estimator which is UMVU over Hölder balls and achieves the (local) minimax rate of convergence. These results hold under weak assumptions on $Φ$ which are satisfied for $Φ(u)=|u|^p$, $p\ge 1$. As an application, we consider the problem of estimating the $L^p$-norm and derive the minimax separation rates…
We introduce a new non parametric method that allows for a direct, fast and efficient estimation of the matrix of kernel norms of a multivariate Hawkes process, also called branching ratio matrix. We demonstrate the capabilities of this method by applying it to high-frequency order book data from the EUREX exchange. We show that it is able to uncover (or recover) various relationships between all the first level order book events associated with some asset when mapped to a 12-dimensional process. We then scale up the model so as to account for events on two assets simultaneously and we discuss…
In this paper we consider the nonparametric estimation of density and regression functions with non-negative support using a gamma kernel procedure introduced by Chen (2000). Strong uniform consistency and asymptotic normality of the corresponding estimators are established under a general ergodic assumption on the data generation process. Our results generalize those of Shi and Song (2016), obtained in the classic i.i.d. framework, and the works of Bouezmarni and Rombouts (2008, 2010b) and Gospodinov and Hirukawa (2007) for mixing time series.
Data Science and AI Research（数据科学与人工智能研究院）；AT&T Chief Data Office（美国电话电报公司首席数据办公室）；University of Minnesota Twin Cities（明尼苏达大学双城分校）
High-dimensional data, where the dimension of the feature space is much larger than sample size, arise in a number of statistical applications. In this context, we construct the generalized multivariate sign transformation, defined as a vector divided by its norm. For different choices of the norm function, the resulting transformed vector adapts to certain geometrical features of the data distribution. Building up on this idea, we obtain one-sample and two-sample testing procedures for mean vectors of high-dimensional data using these generalized sign vectors. These tests are based on U-stati…
We introduce a hierarchical nonparametric model for probability measures based on a multi-resolution transformation of probability distributions. The model allows a varying amount of shrinkage to be applied to data features of different scales and/or at different locations in the sample space, and the varying shrinkage level is locally adaptive to the empirical behavior of the data. Moreover, the model's hierarchical design---through a latent Markov tree structure---allows borrowing of information across locations and scales in setting the adaptive shrinkage level. Inference under the model pr…
In this paper, we consider moderate deviations for Good's coverage estimator. The moderate deviation principle and the self-normalized moderate deviation principle for Good's coverage estimator are established. The results are also applied to the hypothesis testing problem and the confidence interval for the coverage.
In this paper, a nonparametric estimator is proposed for estimating the L1-median for multivariate conditional distribution when the covariates take values in an infinite dimensional space. The multivariate case is more appropriate to predict the components of a vector of random variables simultaneously rather than predicting each of them separately. While estimating the conditional L1-median function using the well-known Nadarya-Waston estimator, we establish the strong consistency of this estimator as well as the asymptotic normality. We also present some simulations and provide how to built…
Consider a Gaussian nonparametric regression problem having both an unknown mean function and unknown variance function. This article presents a class of difference-based kernel estimators for the variance function. Optimal convergence rates that are uniform over broad functional classes and bandwidths are fully characterized, and asymptotic normality is also established. We also show that for suitable asymptotic formulations our estimators achieve the minimax rate.
