In multivariate regression estimation, the rate of convergence depends on the dimension of the regressor. This fact, known as the curse of the dimensionality, motivated several works. The additive model, introduced by Stone (10), offers an efficient response to this problem. In the setting of continuous time processes, using the marginal integration method, we obtain the quadratic convergence rate and the asymptotic normality of the components of the additive model.
We establish asymptotic normality for estimators of the additive regression components under random censorship. To build our estimators, we couple the marginal integration method (Newey (1994)) with an initial Inverse Probability of Censoring Weighted estimator of the multivariate censored regression function introduced by Carbonez et al. (1995) and Kohler et al. (2002). Asymptotic confidence bands are derived from our result.
We establish the asymptotic normality of the regression estimator in a fixed-design setting when the errors are given by a field of dependent random variables. The result applies to martingale-difference or strongly mixing random fields. On this basis, a statistical test that can be applied to image analysis is also presented.
The main purpose of this paper is to estimate the regression function by using a recursive nonparametric kernel approach. We derive the asymptotic normality for a general class of recursive kernel estimate of the regression function, under strong mixing conditions. Our purpose is to extend the work of Roussas and Tran [17] concerning the Devroye-Wagner estimate.
Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstoc…
In this paper, we study the asymptotic normality of the conditional maximum likelihood (ML) estimators for the truncated regression model and the Tobit model. We show that under the general setting assumed in his book, the conjectures made by Hayashi (2000) \footnote{see page 516, and page 520 of Hayashi (2000).} about the asymptotic normality of the conditional ML estimators for both models are true, namely, a sufficient condition is the nonsingularity of $\mathbf{x_tx'_t}$.
Stochastic simulators such as Monte-Carlo estimators are widely used in science and engineering to study physical systems through their probabilistic representation. Global sensitivity analysis aims to identify the input parameters which have the most important impact on the output. A popular tool to perform global sensitivity analysis is the variance-based method which comes from the Hoeffding-Sobol decomposition. Nevertheless, this method requires an important number of simulations and is often unfeasible under reasonable time constraint. Therefore, an approximation of the input/output relat…
This paper is to prove the asymptotic normality of a statistic for detecting the existence of heteroscedasticity for linear regression models without assuming randomness of covariates when the sample size $n$ tends to infinity and the number of covariates $p$ is either fixed or tends to infinity. Moreover our approach indicates that its asymptotic normality holds even without homoscedasticity.
We consider the sparse high-dimensional linear regression model $Y=Xb+ε$ where $b$ is a sparse vector. For the Bayesian approach to this problem, many authors have considered the behavior of the posterior distribution when, in truth, $Y=Xβ+ε$ for some given $β$. There have been numerous results about the rate at which the posterior distribution concentrates around $β$, but few results about the shape of that posterior distribution. We propose a prior distribution for $b$ such that the marginal posterior distribution of an individual coordinate $b_i$ is asymptotically normal centered around an …
In linear models, the generalized least squares (GLS) estimator is applicable when the structure of the error dependence is known. When it is unknown, such structure must be approximated and estimated in a manner that may lead to misspecification. The large-sample analysis of incorrectly-specified GLS (IGLS) estimators requires careful asymptotic manipulations. When performing estimation in the frequency domain, the asymptotic normality of the IGLS estimator, under the so-called Grenander assumptions, has been proved for a broad class of error dependence models. Under the same assumptions, asy…
Nonlinear panel data models with fixed individual effects provide an important set of tools for describing microeconometric data. In a large class of such models (including probit, proportional hazard and quantile regression to name just a few) it is impossible to difference out individual effects, and inference is usually justified in a `large n large T' asymptotic framework. However, there is a considerable gap in the type of assumptions that are currently imposed in models with smooth score functions (such as probit, and proportional hazard) and quantile regression. In the present paper we …
In this paper we study the asymptotic normality in high-dimensional linear regression. We focus on the case where the covariance matrix of the regression variables has a KMS structure, in asymptotic settings where the number of predictors, $p$, is proportional to the number of observations, $n$. The main result of the paper is the derivation of the exact asymptotic distribution for the suitably centered and normalized squared norm of the product between predictor matrix, $\mathbb{X}$, and outcome variable, $Y$, i.e. the statistic $\|\mathbb{X}'Y\|_{2}^{2}$. Additionally, we consider a specific…
This paper discusses asymptotic theory for penalized spline estimators in generalized additive models. The purpose of this paper is to establish the asymptotic bias and variance as well as the asymptotic normality of the penalized spline estimators proposed by Marx and Eilers (1998). Furthermore, the asymptotics for the penalized quasi likelihood fit in mixed models are also discussed.
We show that spline and wavelet series regression estimators for weakly dependent regressors attain the optimal uniform (i.e. sup-norm) convergence rate $(n/\log n)^{-p/(2p+d)}$ of Stone (1982), where $d$ is the number of regressors and $p$ is the smoothness of the regression function. The optimal rate is achieved even for heavy-tailed martingale difference errors with finite $(2+(d/p))$th absolute moment for $d/p<2$. We also establish the asymptotic normality of t statistics for possibly nonlinear, irregular functionals of the conditional mean function under weak conditions. The results are p…
We investigate a semiparametric regression model where one gets noisy non linear non invertible functions of the observations. We focus on the application to bearings-only tracking. We first investigate the least squares estimator and prove its consistency and asymptotic normality under mild assumptions. We study the semiparametric likelihood process and prove local asymptotic normality of the model. This allows to define the efficient Fisher information as a lower bound for the asymptotic variance of regular estimators, and to prove that the parametric likelihood estimator is regular and asym…
Sensitivity analysis of the optimal solution in response surface methodology is studied and an explicit form of the effect of perturbation of the regression coefficients on the optimal solution is obtained. The characterisation of the critical point of the convex program corresponding to the optimum of a response surface model is also studied. The asymptotic normality of the optimal solution follows by standard methods.
We establish the asymptotic normality of the kernel type estimator for the regression function constructed from quasi-associated data when the explanatory variable takes its values in a separable Hilbert space.
We revisit the model of heteroscedastic extremes initially introduced by Einmahl et al. (JRSSB, 2016) to describe the evolution of a non stationary sequence whose extremes evolve over time and adapt it into a general extreme quantile regression framework. We provide estimates for the extreme value index and the integrated skedasis function and prove their asymptotic normality. Our results are quite similar to those developed for heteroscedastic extremes but with a different proof approach emphasizing coupling arguments. We also propose a pointwise estimator of the skedasis function and a Weiss…
We study minimax rates for high-dimensional linear regression with additive errors under the $\ell_p\ (1\leq p<\infty)$-losses, where the regression parameter is of weak sparsity. Our lower and upper bounds agree up to constant factors, implying that the proposed estimator is minimax optimal.
In this work is obtained an explicit form for the perturbation effect on the matrix of regression coefficients on the optimal solution in multiresponse surface methodology. Then, the sensitivity analysis of the optimal solution is studied and the critical point characterisation of the convex program, associated with the optimum of a multiresponse surface, is also analysed. Finally, the asymptotic normality of the optimal solution is derived by standard methods.
This paper deals with the estimation of hidden periodicities in a non-linear regression model with stationary noise displaying cyclical dependence. Consistency and asymptotic normality are established for the least-squares estimates.
This paper deals with the Fisher-consistency, weak continuity and differentiability of estimating functionals corresponding to a class of both linear and nonlinear regression high breakdown M estimates, which includes S and MM estimates. A restricted type of differentiability, called weak differentiability, is defined, which suffices to prove the asymptotic normality of estimates based on the functionals. This approach allows to prove the consistency, asymptotic normality and qualitative robustness of M estimates under more general conditions than those required in standard approaches. In part…
In a recent paper Birke and Bissantz (2008) considered the problem of nonparametric estimation in inverse regression models with convolution-type operators. For multivariate predictors nonparametric methods suffer from the curse of dimensionality and we consider inverse regression models with the additional qualitative assumption of additivity. In these models several additive estimators are studied. In particular, we investigate estimators under the random design assumption which are applicable when observations are not available on a grid. Finally, we compare this estimator with the marginal…
We consider estimation in a sparse additive regression model with the design points on a regular lattice. We establish the minimax convergence rates over Sobolev classes and propose a Fourier-based rate-optimal estimator which is adaptive to the unknown sparsity and smoothness of the response function. The estimator is derived within Bayesian formalism but can be naturally viewed as a penalized maximum likelihood estimator with the complexity penalties on the number of nonzero univariate additive components of the response and on the numbers of the nonzero coefficients of their Fourer expansio…
This short note is to point the reader to notice that the proof of high dimensional asymptotic normality of MLE estimator for logistic regression under the regime $p_n=o(n)$ given in paper: "Maximum likelihood estimation in logistic regression models with a diverging number of covariates. Electronic Journal of Statistics, 6, 1838-1846." is wrong.
In this study, we develop an asymptotic theory of nonparametric regression for locally stationary random fields (LSRFs) $\{{\bf X}_{{\bf s}, A_{n}}: {\bf s} \in R_{n} \}$ in $\mathbb{R}^{p}$ observed at irregularly spaced locations in $R_{n} =[0,A_{n}]^{d} \subset \mathbb{R}^{d}$. We first derive the uniform convergence rate of general kernel estimators, followed by the asymptotic normality of an estimator for the mean function of the model. Moreover, we consider additive models to avoid the curse of dimensionality arising from the dependence of the convergence rate of estimators on the number…
This paper is concerned with asymptotic theory for penalized spline estimator in bivariate additive model. The focus of this paper is put upon the penalized spline estimator obtained by the backfitting algorithm. The convergence of the algorithm as well as the uniqueness of its solution are shown. The asymptotic bias and variance of penalized spline estimator are derived by an efficient use of the asymptotic results for the penalized spline estimator in marginal univariate model. Asymptotic normality of estimator is also developed, by which an approximate confidence interval can be obtained. S…
We propose a new testing procedure of heteroskedasticity in high-dimensional linear regression, where the number of covariates can be larger than the sample size. Our testing procedure is based on residuals of the Lasso. We demonstrate that our test statistic has asymptotic normality under the null hypothesis of homoskedasticity. Simulation results show that the proposed testing procedure obtains accurate empirical sizes and powers. We also present results of real economic data applications.
Additive models and generalized additive models are effective semiparametric tools for multidimensional data. In this article we propose an online smoothing backfitting method for generalized additive models with local polynomial smoothers. The main idea is to use a second order expansion to approximate the nonlinear integral equations to maximize the local quasilikelihood and store the coefficients as the sufficient statistics which can be updated in an online manner by a dynamic candidate bandwidth method. The updating procedure only depends on the stored sufficient statistics and the curren…
This paper deals with the nonparametric density estimation of the regression error term assuming its independence with the covariate. The difference between the feasible estimator which uses the estimated residuals and the unfeasible one using the true residuals is studied. An optimal choice of the bandwidth used to estimate the residuals is given. We also study the asymptotic normality of the feasible kernel estimator and its rate-optimality.
Under a partially linear models we study a family of robust estimates for the regression parameter and the regression function when some of the predictor variables take values on a Riemannian manifold. We obtain the consistency and the asymptotic normality of the proposed estimators. Also, we consider a robust cross validation procedure to select the smoothing parameter. Simulations and application to real data show the performance of our proposal under small samples and contamination.
We use local polynomial fitting to estimate the nonparametric M-regression function for strongly mixing stationary processes $\{(Y_{i},\underline{X}_{i})\}$. We establish a strong uniform consistency rate for the Bahadur representation of estimators of the regression function and its derivatives. These results are fundamental for statistical inference and for applications that involve plugging in such estimators into other functionals where some control over higher order terms are required. We apply our results to the estimation of an additive M-regression model.
We consider linear processes, not necessarily Gaussian, with long, short or negative memory. The memory parameter is estimated semi-parametrically using wavelets from a sample $X_1,...,X_n$ of the process. We treat both the log-regression wavelet estimator and the wavelet Whittle estimator. We show that these estimators are asymptotically normal as the sample size $n\to\infty$ and we obtain an explicit expression for the limit variance. These results are derived from a general result on the asymptotic normality of the empirical scalogram for linear processes, conveniently centered and normaliz…
This paper is devoted to the nonparametric estimation of the derivative of the regression function in a nonparametric regression model. We implement a very efficient and easy to handle statistical procedure based on the derivative of the recursive Nadaraya-Watson estimator. We establish the almost sure convergence as well as the asymptotic normality for our estimates. We also illustrate our nonparametric estimation procedure on simulated and real life data associated with sea shores water quality and valvometry.
We prove conditional asymptotic normality of a class of quadratic U-statistics that are dominated by their degenerate second order part and have kernels that change with the number of observations. These statistics arise in the construction of estimators in high-dimensional semi- and non-parametric models, and in the construction of nonparametric confidence sets. This is illustrated by estimation of the integral of a square of a density or regression function, and estimation of the mean response with missing data. We show that estimators are asymptotically normal even in the case that the rate…
We study asymptotic behavior of one-step weighted $M$-estimators based on samples from arrays of not necessarily identically distributed random variables and representing explicit approximations to the corresponding consistent weighted $M$-estimators. Sufficient conditions are presented for asymptotic normality of the one-step weighted $M$-estimators under consideration. As a consequence, we consider some well-known nonlinear regression models where the procedure mentioned allow us to construct explicit asymptotically optimal estimators.
A parameter estimation problem is considered for a diagonaliazable stochastic evolution equation using a finite number of the Fourier coefficients of the solution. The equation is driven by additive noise that is white in space and fractional in time with the Hurst parameter $H\geq 1/2$. The objective is to study asymptotic properties of the maximum likelihood estimator as the number of the Fourier coefficients increases. A necessary and sufficient condition for consistency and asymptotic normality is presented in terms of the eigenvalues of the operators in the equation.
A continuous-time nonlinear regression model with Lévy-driven linear noise process is considered. Sufficient conditions of consistency and asymptotic normality of the Whittle estimator for the parameter of the noise spectral density are obtained in the paper.
In this article, we consider flexible seasonal time series models which consist of a common trend function over periods and additive individual trend (seasonal effect) functions. The consistency and asymptotic normality of the local linear estimators were obtained under the $α$-mixing conditions and without specifying the error distribution. We develop these results to consistency and asymptotic normality of local linear estimates by using central limit theorems for flexible seasonal time series model, which error terms are $k$-weak dependent and $λ$-weak dependent random variables.
In this paper, based on the kernel estimator proposed by Ould-Said and Lemdani (Ann. Instit. Statist. Math. 2006), we develop some new generalized M-estimator procedures for single index regression models with left-truncated responses. The consistency and asymptotic normality of our estimators are also established. Some simulation studies are given to investigate the finite sample performance of the proposed estimators.
We study theoretical properties of regularized robust M-estimators, applicable when data are drawn from a sparse high-dimensional linear model and contaminated by heavy-tailed distributions and/or outliers in the additive errors and covariates. We first establish a form of local statistical consistency for the penalized regression estimators under fairly mild conditions on the error distribution: When the derivative of the loss function is bounded and satisfies a local restricted curvature condition, all stationary points within a constant radius of the true regression vector converge at the m…
To fast approximate maximum likelihood estimators with massive data, this paper studies the Optimal Subsampling Method under the A-optimality Criterion (OSMAC) for generalized linear models. The consistency and asymptotic normality of the estimator from a general subsampling algorithm are established, and optimal subsampling probabilities under the A- and L-optimality criteria are derived. Furthermore, using Frobenius norm matrix concentration inequalities, finite sample properties of the subsample estimator based on optimal subsampling probabilities are also derived. Since the optimal subsamp…
We develop uniformly valid confidence regions for regression coefficients in a high-dimensional sparse median regression model with homoscedastic errors. Our methods are based on a moment equation that is immunized against non-regular estimation of the nuisance part of the median regression function by using Neyman's orthogonalization. We establish that the resulting instrumental median regression estimator of a target regression coefficient is asymptotically normally distributed uniformly with respect to the underlying sparse model and is semi-parametrically efficient. We also generalize our …
In this paper, we consider an unknown functional estimation problem in a general nonparametric regression model with the feature of having both multiplicative and additive noise.We propose two new wavelet estimators in this general context. We prove that they achieve fast convergence rates under the mean integrated square error over Besov spaces. The obtained rates have the particularity of being established under weak conditions on the model. A numerical study in a context comparable to stochastic frontier estimation (with the difference that the boundary is not necessarily a production funct…
In this paper we study the problem of estimating the drift/viscosity coefficient for a large class of linear, parabolic stochastic partial differential equations (SPDEs) driven by an additive space-time noise. We propose a new class of estimators, called trajectory fitting estimators (TFEs). The estimators are constructed by fitting the observed trajectory with an artificial one, and can be viewed as an analog to the classical least squares estimators from the time-series analysis. As in the existing literature on statistical inference for SPDEs, we take a spectral approach, and assume that we…
We present a new method for estimating the frontier of a multidimensional sample. The estimator is based on a kernel regression on the power-transformed data. We assume that the exponent of the transformation goes to infinity while the bandwidth of the kernel goes to zero. We give conditions on these two parameters to obtain complete convergence and asymptotic normality. The good performance of the estimator is illustrated on some finite sample situations.
We consider a resampling scheme for parameters estimates in nonlinear regression models. We provide an estimation procedure which recycles, via random weighting, the relevant parameters estimates to construct consistent estimates of the sampling distribution of the various estimates. We establish the asymptotic normality of the resampled estimates and demonstrate the applicability of the recycling approach in a small simulation study and via example.
Additive models play an essential role in studying non-linear relationships. Despite many recent advances in estimation, there is a lack of methods and theories for inference in high-dimensional additive models, including confidence interval construction and hypothesis testing. Motivated by inference for non-linear treatment effects, we consider the high-dimensional additive model and make inference for the derivative of the function of interest. We propose a novel decorrelated local linear estimator and establish its asymptotic normality. The main novelty is the construction of the decorrelat…
Estimating the innovation probability density is an important issue in any regression analysis. This paper focuses on functional autoregressive models. A residual-based kernel estimator is proposed for the innovation density. Asymptotic properties of this estimator depend on the average prediction error of the functional autoregressive function. Sufficient conditions are studied to provide strong uniform consistency and asymptotic normality of the kernel density estimator.
Rutgers University（罗格斯大学）；Voleon Group（Voleon集团）；DeepMind（深度思维）；Microsoft Research（微软研究院）；Massachusetts Institute of Technology（麻省理工学院）；UC Berkeley（加州大学伯克利分校）
When data is collected in an adaptive manner, even simple methods like ordinary least squares can exhibit non-normal asymptotic behavior. As an undesirable consequence, hypothesis tests and confidence intervals based on asymptotic normality can lead to erroneous results. We propose a family of online debiasing estimators to correct these distributional anomalies in least squares estimation. Our proposed methods take advantage of the covariance structure present in the dataset and provide sharper estimates in directions for which more information has accrued. We establish an asymptotic normalit…
Segmented regression is a standard statistical procedure used to estimate the effect of a policy intervention on time series outcomes. This statistical method assumes the normality of the outcome variable, a large sample size, no autocorrelation in the observations, and a linear trend over time. Also, segmented regression is very sensitive to outliers. In a small sample study, if the outcome variable does not follow a Gaussian distribution, then using segmented regression to estimate the intervention effect leads to incorrect inferences. To address the small sample problem and non-normality in…
Additive models are flexible regression tools that handle linear as well as nonlinear terms. The latter are typically modelled via smoothing splines. Additive mixed models extend additive models to include random terms when the data are sampled according to cluster designs (e.g., longitudinal). These models find applications in the study of phenomena like growth, certain disease mechanisms and energy consumption in humans, when repeated measurements are available. In this paper, we propose a novel additive mixed model for quantile regression. Our methods are motivated by an application to phys…
