This paper discusses asymptotic distributions of various estimators of the underlying parameters in some regression models with long memory (LM) Gaussian design and nonparametric heteroscedastic LM moving average errors. In the simple linear regression model, the first-order asymptotic distribution of the least square estimator of the slope parameter is observed to be degenerate. However, in the second order, this estimator is $n^{1/2}$-consistent and asymptotically normal for $h+H<3/2$; nonnormal otherwise, where $h$ and $H$ are LM parameters of design and error processes, respectively. The finite-dimensional asymptotic distributions of a class of kernel type estimators of the conditional variance function $σ^2(x)$ in a more general heteroscedastic regression model are found to be normal whenever $H<(1+h)/2$, and non-normal otherwise. In addition, in this general model, $\log(n)$-consistency of the local Whittle estimator of $H$ based on pseudo residuals and consistency of a cross validation type estimator of $σ^2(x)$ are established. All of these findings are then used to propose a lack-of-fit test of a parametric regression model, with an application to some currency exchange rate data which exhibit LM.
An adaptive nonparametric estimation procedure is constructed for heteroscedastic regression when the noise variance depends on the unknown regression. A non-asymptotic upper bound for a quadratic risk (oracle inequality) is obtained
An adaptive nonparametric estimation procedure is constructed for the estimation problem of heteroscedastic regression when the noise variance depends on the unknown regression. A non-asymptotic upper bound for a quadratic risk (an oracle inequality) is constructed.
This paper concerns the estimation of the regression function at a given point in nonparametric heteroscedastic models with Gaussian noise or with noise having unknown distribution. In the two cases an asymptotically efficient kernel estimator is constructed for the minimax absolute error risk.
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper (2007) for estimation of unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk. It means that the asymptotic quadratic risk for this procedure coincides with a sharp lower bound.
Heteroscedasticity is common in real world applications and is often handled by incorporating case weights into a modeling procedure. Intuitively, models fitted with different weight schemes would have a different level of complexity depending on how well the weights match the inverse of error variances. However, existing statistical theories on model complexity, also known as model degrees of freedom, were primarily established under the assumption of equal error variances. In this work, we focus on linear regression procedures and seek to extend the existing measures to a heteroscedastic set…
This paper provides general expression for Bartlett and Bartlett-type correction factors for the likelihood ratio and gradient statistics to test the dispersion parameter in heteroscedastic symmetric nonlinear models. This class of regression models is potentially useful for modeling data containing outlying observations. We consider a partition on the dispersion parameter vector in order to test the parameters of interest. Furthermore, we develop Monte Carlo simulations to compare the finite sample performances of the corrected tests proposed with the usual and modified score tests, likelihoo…
We investigate function estimation in nonparametric regression models with random design and heteroscedastic correlated noise. Adaptive properties of warped wavelet nonlinear approximations are studied over a wide range of Besov scales, $f\in\mathcal{B}^s_{π,r}$, and for a variety of $L^p$ error measures. We consider error distributions with Long-Range-Dependence parameter $α,0<α\leq1$; heteroscedasticity is modeled with a design dependent function $σ$. We prescribe a tuning paradigm, under which warped wavelet estimation achieves partial or full adaptivity results with the rates that are show…
We revisit the model of heteroscedastic extremes initially introduced by Einmahl et al. (JRSSB, 2016) to describe the evolution of a non stationary sequence whose extremes evolve over time and adapt it into a general extreme quantile regression framework. We provide estimates for the extreme value index and the integrated skedasis function and prove their asymptotic normality. Our results are quite similar to those developed for heteroscedastic extremes but with a different proof approach emphasizing coupling arguments. We also propose a pointwise estimator of the skedasis function and a Weiss…
In this paper we consider a heteroscedastic transformation model, where the transformation belongs to a parametric family of monotone transformations, the regression and variance function are modelled nonparametrically and the error is independent of the multidimensional covariates. In this model, we first consider the estimation of the unknown components of the model, namely the transformation parameter, regression and variance function and the distribution of the error. We show the asymptotic normality of the proposed estimators. Second, we propose tests for the validity of the model, and es…
In this paper we propose a new test of heteroscedasticity for parametric regression models and partial linear regression models in high dimensional settings. When the dimension of covariates is large, existing tests of heteroscedasticity perform badly due to the \curse of dimensionality". To attack this problem, we construct a test of heteroscedasticity by using a projection-based empirical process. We study the asymptotic properties of the test statistic under the null hypothesis and alternative hypotheses. It is shown that the test can detect local alternatives departure from the null hypoth…
We consider a heteroscedastic regression model in which some of the regression coefficients are zero but it is not known which ones. Penalized quantile regression is a useful approach for analyzing such data. By allowing different covariates to be relevant for modeling conditional quantile functions at different quantile levels, it provides a more complete picture of the conditional distribution of a response variable than mean regression. Existing work on penalized quantile regression has been mostly focused on point estimation. Although bootstrap procedures have recently been shown to be eff…
Heteroscedasticity testing is of importance in regression analysis. Existing local smoothing tests suffer severely from curse of dimensionality even when the number of covariates is moderate because of use of nonparametric estimation. In this paper, a dimension reduction-based model adaptive test is proposed which behaves like a local smoothing test as if the number of covariates were equal to the number of their linear combinations in the mean regression function, in particular, equal to 1 when the mean function contains a single index. The test statistic is asymptotically normal under the nu…
This paper considers quantile regression for a wide class of time series models including ARMA models with asymmetric GARCH (AGARCH) errors. The classical mean-variance models are reinterpreted as conditional location-scale models so that the quantile regression method can be naturally geared into the considered models. The consistency and asymptotic normality of the quantile regression estimator is established in location-scale time series models under mild conditions. In the application of this result to ARMA-AGARCH models, more primitive conditions are deduced to obtain the asymptotic prope…
Testing heteroscedasticity of the errors is a major challenge in high-dimensional regressions where the number of covariates is large compared to the sample size. Traditional procedures such as the White and the Breusch-Pagan tests typically suffer from low sizes and powers. This paper proposes two new test procedures based on standard OLS residuals. Using the theory of random Haar orthogonal matrices, the asymptotic normality of both test statistics is obtained under the null when the degree of freedom tends to infinity. This encompasses both the classical low-dimensional setting where the nu…
Estimating conditional quantiles of financial time series is essential for risk management and many other applications in finance. It is well-known that financial time series display conditional heteroscedasticity. Among the large number of conditional heteroscedastic models, the generalized autoregressive conditional heteroscedastic (GARCH) process is the most popular and influential one. So far, feasible quantile regression methods for this task have been confined to a variant of the GARCH model, the linear GARCH model, owing to its tractable conditional quantile structure. This paper consid…
This paper is to prove the asymptotic normality of a statistic for detecting the existence of heteroscedasticity for linear regression models without assuming randomness of covariates when the sample size $n$ tends to infinity and the number of covariates $p$ is either fixed or tends to infinity. Moreover our approach indicates that its asymptotic normality holds even without homoscedasticity.
The problem of estimating the baseline signal from multisample noisy curves is investigated. We consider the functional mixed effects model, and we suppose that the functional fixed effect belongs to the Besov class. This framework allows us to model curves that can exhibit strong irregularities, such as peaks or jumps for instance. The lower bound for the $L_2$ minimax risk is provided, as well as the upper bound of the minimax rate, that is derived by constructing a wavelet estimator for the functional fixed effect. Our work constitutes the first theoretical functional results in multisample…
Estimation for the log-logistic and Weibull distributions can be performed by using the equations used for probability plotting. The equations leads to highly heteroscedastic regression. Exact expressions for the variances of the residuals are derived which can be used to perform weighted regression. In large samples maximum likelihood performs best, but it is shown that in smaller samples the weighted regression outperforms maximum likelihood estimation with respect to bias and mean square error.
The paper is devoted to the problem of estimation of a univariate component in a heteroscedastic nonparametric multiple regression under the mean integrated squared error (MISE) criteria. The aim is to understand how the scale function should be used for estimation of the univariate component. It is known that the scale function does not affect the rate of the MISE convergence, and as a result sharp constants are explored. The paper begins with developing a sharp-minimax theory for a pivotal model $Y=f(X)+σ(X,\mathbf{Z})\varepsilon$, where $\varepsilon$ is standard normal and independent of th…
Scalable regression calibration approaches to correcting measurement error in multi-level generalized functional linear regression models with heteroscedastic measurement errors
We study the problem of online generalized linear regression in the stochastic setting, where the label is generated from a generalized linear model with possibly unbounded additive noise. We provide a sharp analysis of the classical follow-the-regularized-leader (FTRL) algorithm to cope with the label noise. More specifically, for $σ$-sub-Gaussian label noise, our analysis provides a regret upper bound of $O(σ^2 d \log T) + o(\log T)$, where $d$ is the dimension of the input vector, $T$ is the total number of rounds. We also prove a $Ω(σ^2d\log(T/d))$ lower bound for stochastic online linear …
Many time series exhibit changes both in level and in variability. Generally, it is more important to detect a change in the level, and changing or smoothly evolving variability can confound existing tests. This paper develops a framework for testing for shifts in the level of a series which accommodates the possibility of changing variability. The resulting tests are robust both to heteroskedasticity and serial dependence. They rely on a new functional central limit theorem for dependent random variables whose variance can change or trend in a substantial way. This new result is of independen…
Concerning bivariate least squares linear regression, the classical approach pursued for functional models in earlier attempts is reviewed using a new formalism in terms of deviation (matrix) traces. Within the framework of classical error models, the dependent variable relates to the independent variable according to the usual additive model. Linear models of regression lines are considered in the general case of correlated errors in X and in Y for heteroscedastic data. The special case of (C) generalized orthogonal regression is considered in detail together with well known subcases. In the …
This paper deals with the nonparametric estimation in heteroscedastic regression $ Y_i=f(X_i)+ξ_i, \: i=1,...,n $, with incomplete information, i.e. each real random variable $ ξ_i $ has a density $ g_{i} $ which is unknown to the statistician. The aim is to estimate the regression function $ f $ at a given point. Using a local polynomial fitting from M-estimator denoted $ \hat f^h $ and applying Lepski's procedure for the bandwidth selection, we construct an estimator $ \hat f^{\hat h} $ which is adaptive over the collection of isotropic Hölder classes. In particular, we establish new exponen…
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating a unknown nonparametric regression. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimators.
We consider the problem of choosing between several models in least-squares regression with heteroscedastic data. We prove that any penalization procedure is suboptimal when the penalty is a function of the dimension of the model, at least for some typical heteroscedastic model selection problems. In particular, Mallows' Cp is suboptimal in this framework. On the contrary, optimal model selection is possible with data-driven penalties such as resampling or $V$-fold penalties. Therefore, it is worth estimating the shape of the penalty from data, even at the price of a higher computational cost.…
The paper deals with asymptotic properties of the adaptive procedure proposed in the author paper, 2007, for estimating an unknown nonparametric regression. %\cite{GaPe1}. We prove that this procedure is asymptotically efficient for a quadratic risk, i.e. the asymptotic quadratic risk for this procedure coincides with the Pinsker constant which gives a sharp lower bound for the quadratic risk over all possible estimates
We consider the estimation of a regression function with random design and heteroscedastic noise in a nonparametric setting. More precisely, we address the problem of characterizing the optimal penalty when the regression function is estimated by using a penalized least-squares model selection method. In this context, we show the existence of a minimal penalty, defined to be the maximum level of penalization under which the model selection procedure totally misbehaves. The optimal penalty is shown to be twice the minimal one and to satisfy a non-asymptotic pathwise oracle inequality with leadi…
Concerning bivariate least squares linear regression, the classical results obtained for extreme structural models in earlier attempts are reviewed using a new formalism in terms of deviation (matrix) traces which, for homoscedastic data, reduce to usual quantities leaving aside an unessential (but dimensional) multiplicative factor. Within the framework of classical error models, the dependent variable relates to the independent variable according to the usual additive model. The classes of linear models considered are regression lines in the limit of uncorrelated errors in X and in Y. For ho…
We give methods for the construction of designs for linear models, when the purpose of the investigation is the estimation of the conditional quantile function and the estimation method is quantile regression. The designs are robust against misspecified response functions, and against unanticipated heteroscedasticity. The methods are illustrated by example, and in a case study in which they are applied to growth charts.
We consider the estimation of a bounded regression function with nonparametric heteroscedastic noise and random design. We study the true and empirical excess risks of the least-squares estimator on finite-dimensional vector spaces. We give upper and lower bounds on these quantities that are nonasymptotic and optimal to first order, allowing the dimension to depend on sample size. These bounds show the equivalence between the true and empirical excess risks when, among other things, the least-squares estimator is consistent in sup-norm with the projection of the regression function onto the co…
We consider a wavelet thresholding approach to adaptive variance function estimation in heteroscedastic nonparametric regression. A data-driven estimator is constructed by applying wavelet thresholding to the squared first-order differences of the observations. We show that the variance function estimator is nearly optimally adaptive to the smoothness of both the mean and variance functions. The estimator is shown to achieve the optimal adaptive rate of convergence under the pointwise squared error simultaneously over a range of smoothness classes. The estimator is also adaptively within a log…
Quantile regression is an important tool for estimation of conditional quantiles of a response Y given a vector of covariates X. It can be used to measure the effect of covariates not only in the center of a distribution, but also in the upper and lower tails. This paper develops a theory of quantile regression in the tails. Specifically, it obtains the large sample properties of extremal (extreme order and intermediate order) quantile regression estimators for the linear quantile regression model with the tails restricted to the domain of minimum attraction and closed under tail equivalence a…
We tackle the problem of building a prediction interval in heteroscedastic Gaussian regression. We focus on prediction intervals with constrained expected length in order to guarantee interpretability of the output. In this framework, we derive a closed form expression of the optimal prediction interval that allows for the development a data-driven prediction interval based on plug-in. The construction of the proposed algorithm is based on two samples, one labeled and another unlabeled. Under mild conditions, we show that our procedure is asymptotically as good as the optimal prediction interv…
We prove a new and general concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise. No specific structure is required on the model, except the existence of a suitable function that controls the local suprema of the empirical process. So far, only the case of linear contrast estimation was tackled in the literature with this level of generality on the model. We solve here the case of a quadratic contrast, by separating the behavior of a linearized empirical process and the empirical process driven by the squares of functions of model…
The paper concerns the problem of pointwise adaptive estimation in regression when the noise is heteroscedastic and incorrectly known. The use of the local approximation method, which includes the local polynomial smoothing as a particular case, leads to a finite family of estimators corresponding to different degrees of smoothing. Data-driven choice of localization degree in this case can be understood as the problem of selection from this family. This task can be performed by a suggested in Katkovnik and Spokoiny (2008) FLL technique based on Lepski's method. An important issue with this typ…
If uncorrelated random variables have a common expected value and decreasing variances then the variance of a sample mean is decreasing with the number of observations. Unfortunately, this natural and desirable Variance Reduction Property (VRP) by augmenting data is not automatically inherited by Ordinary Least Squares (OLS) estimators of parameters. In the paper we find conditions for the OLS to have the VRP. In the case of a straight line regression we show that the OLS estimators of intercept and slope have the VRP if the design points are increasing. This also holds true for alternating tw…
High-dimensional data can often display heterogeneity due to heteroscedastic variance or inhomogeneous covariate effects. Penalized quantile and expectile regression methods offer useful tools to detect heteroscedasticity in high-dimensional data. The former is computationally challenging due to the non-smooth nature of the check loss, and the latter is sensitive to heavy-tailed error distributions. In this paper, we propose and study (penalized) robust expectile regression (retire), with a focus on iteratively reweighted $\ell_1$-penalization which reduces the estimation bias from $\ell_1$-pe…
We discuss a class of conditionally heteroscedastic time series models satisfying the equation $r_t= ζ_t σ_t$, where $ζ_t$ are standardized i.i.d. r.v.'s and the conditional standard deviation $σ_t$ is a nonlinear function $Q$ of inhomogeneous linear combination of past values $r_s, s<t$ with coefficients $b_j$. The existence of stationary solution $r_t$ with finite $p$th moment, $0< p < \infty $ is obtained under some conditions on $Q, b_j$ and $p$th moment of $ζ_0$. Weak dependence properties of $r_t$ are studied, including the invariance principle for partial sums of Lipschitz functions of …
The ultimate goal of regression analysis is to obtain information about the conditional distribution of a response given a set of explanatory variables. This goal is, however, seldom achieved because most established regression models only estimate the conditional mean as a function of the explanatory variables and assume that higher moments are not affected by the regressors. The underlying reason for such a restriction is the assumption of additivity of signal and noise. We propose to relax this common assumption in the framework of transformation models. The novel class of semiparametric re…
We consider the common nonlinear regression model where the variance, as well as the mean, is a parametric function of the explanatory variables. The $c$-optimal design problem is investigated in the case when the parameters of both the mean and the variance function are of interest. A geometric characterization of $c$-optimal designs in this context is presented, which generalizes the classical result of Elfving [Ann. Math. Statist. 23 (1952) 255--262] for $c$-optimal designs. As in Elfving's famous characterization, $c$-optimal designs can be described as representations of boundary points o…
Conformal prediction is a technique for constructing prediction intervals that attain valid coverage in finite samples, without making distributional assumptions. Despite this appeal, existing conformal methods can be unnecessarily conservative because they form intervals of constant or weakly varying length across the input space. In this paper we propose a new method that is fully adaptive to heteroscedasticity. It combines conformal prediction with classical quantile regression, inheriting the advantages of both. We establish a theoretical guarantee of valid coverage, supplemented by extens…
