We consider a nonlinear polynomial regression model in which we wish to test the null hypothesis of structural stability in the regression parameters against the alternative of a break at an unknown time. We derive the extreme value distribution of a maximum-type test statistic which is asymptotically equivalent to the maximally selected likelihood ratio. The resulting test is easy to apply and has good size and power, even in small samples.
We consider the related problems of estimating the $l_2$-norm and the squared $l_2$-norm in sparse linear regression with unknown variance, as well as the problem of testing the hypothesis that the regression parameter is null under sparse alternatives with $l_2$ separation. We establish the minimax optimal rates of estimation (respectively, testing) in these three problems.
We propose a new testing procedure of heteroskedasticity in high-dimensional linear regression, where the number of covariates can be larger than the sample size. Our testing procedure is based on residuals of the Lasso. We demonstrate that our test statistic has asymptotic normality under the null hypothesis of homoskedasticity. Simulation results show that the proposed testing procedure obtains accurate empirical sizes and powers. We also present results of real economic data applications.
In this paper, we are concerned with nonparametric estimation of the multivariate regression function in the presence of right censored data. More precisely, we propose a statistic that is shown to be asymptotically normally distributed under the additive assumption, and that could be used to test for additivity in the censored regression setting.
We consider the problem of testing the hypothesis that the parameter of linear regression model is 0 against an s-sparse alternative separated from 0 in the l2-distance. We show that, in Gaussian linear regression model with p < n, where p is the dimension of the parameter and n is the sample size, the non-asymptotic minimax rate of testing has the form sqrt((s/n) log(1 + sqrt(p)/s )). We also show that this is the minimax rate of estimation of the l2-norm of the regression parameter.
We propose a novel test statistic for testing exogeneity in the functional linear regression model. In contrast to Hausman-type tests in finite dimensional linear regression setups, a direct extension to the functional linear regression model is not possible. Instead, we propose a test statistic based on the sum of the squared difference of projections of the two estimators for testing the null hypothesis of exogeneity in the functional linear regression model. We derive asymptotic normality under the null and consistency under general alternatives. Moreover, we prove bootstrap consistency res…
In this paper we propose a general methodology, based on multiple testing, for testing that the mean of a Gaussian vector in R^n belongs to a convex set. We show that the test achieves its nominal level, and characterize a class of vectors over which the tests achieve a prescribed power. In the functional regression model this general methodology is applied to test some qualitative hypotheses on the regression function. For example, we test that the regression function is positive, increasing, convex, or more generally, satisfies a differential inequality. Uniform separation rates over classes…
Recently a distribution free approach for testing parametric hypotheses based on unitary transformations has been suggested in \cite{Khm13, Khm16, Khm17} and further studied in \cite{Ngu17} and \cite{Rob19}. In this note we show that the transformation takes extremely simple form in distribution free testing of linear regression. Then we extend it to general parametric regression with vector-valued covariates.
We consider a regression model with errors that are a.s. negative. Thus the regression function is not the expected value of the observations but the right endpoint of their support. We develop two goodness-of-fit tests for the hypotheses that the regression function is an affine function, study the asymptotic distributions of the test statistics in order to approximately fix the sizes of the tests, derive their finite-sample properties based on simulations and apply them to life expectancy data.
We consider the problem of inference on a regression function at a point when the entire function satisfies a sign or shape restriction under the null. We propose a test that achieves the optimal minimax rate adaptively over a range of Hölder classes, up to a $\log\log n$ term, which we show to be necessary for adaptation. We apply the results to adaptive one-sided tests for the regression discontinuity parameter under a monotonicity restriction, the value of a monotone regression function at the boundary and the proportion of true null hypotheses in a multiple testing problem.
In this paper, we establish a uniform error rate of a Bahadur representation for local polynomial estimators of quantile regression functions. The error rate is uniform over a range of quantiles, a range of evaluation points in the regressors, and over a wide class of probabilities for observed random variables. Most of the existing results on Bahadur representations for local polynomial quantile regression estimators apply to the fixed data generating process. In the context of testing monotonicity where the null hypothesis is of a complex composite hypothesis, it is particularly relevant to …
In this paper we propose a new test for the hypothesis of a constant coefficient of variation in the common nonparametric regression model. The test is based on an estimate of the $L^2$-distance between the square of the regression function and variance function. We prove asymptotic normality of a standardized estimate of this distance under the null hypothesis and fixed alternatives and the finite sample properties of a corresponding bootstrap test are investigated by means of a simulation study. The results are applicable to stationary processes with the common mixing conditions and are used…
We discuss nonparametric tests for parametric specifications of regression quantiles. The test is based on the comparison of parametric and nonparametric fits of these quantiles. The nonparametric fit is a Nadaraya-Watson quantile smoothing estimator. An asymptotic treatment of the test statistic requires the development of new mathematical arguments. An approach that makes only use of plugging in a Bahadur expansion of the nonparametric estimator is not satisfactory. It requires too strong conditions on the dimension and the choice of the bandwidth. Our alternative mathematical approach requi…
We consider the problem of sparsity testing in the high-dimensional linear regression model. The problem is to test whether the number of non-zero components (aka the sparsity) of the regression parameter $θ^*$ is less than or equal to $k_0$. We pinpoint the minimax separation distances for this problem, which amounts to quantifying how far a $k_1$-sparse vector $θ^*$ has to be from the set of $k_0$-sparse vectors so that a test is able to reject the null hypothesis with high probability. Two scenarios are considered. In the independent scenario, the covariates are i.i.d. normally distributed …
We consider the problem of testing a particular type of composite null hypothesis under a nonparametric multivariate regression model. For a given quadratic functional $Q$, the null hypothesis states that the regression function $f$ satisfies the constraint $Q[f]=0$, while the alternative corresponds to the functions for which $Q[f]$ is bounded away from zero. On the one hand, we provide minimax rates of testing and the exact separation constants, along with a sharp-optimal testing procedure, for diagonal and nonnegative quadratic functionals. We consider smoothness classes of ellipsoidal form…
High-dimensional logistic regression is widely used in analyzing data with binary outcomes. In this paper, global testing and large-scale multiple testing for the regression coefficients are considered in both single- and two-regression settings. A test statistic for testing the global null hypothesis is constructed using a generalized low-dimensional projection for bias correction and its asymptotic null distribution is derived. A lower bound for the global testing is established, which shows that the proposed test is asymptotically minimax optimal over some sparsity range. For testing the in…
This paper presents a selective survey of recent developments in statistical inference and multiple testing for high-dimensional regression models, including linear and logistic regression. We examine the construction of confidence intervals and hypothesis tests for various low-dimensional objectives such as regression coefficients and linear and quadratic functionals. The key technique is to generate debiased and desparsified estimators for the targeted low-dimensional objectives and estimate their uncertainty. In addition to covering the motivations for and intuitions behind these statistica…
The partial copula provides a method for describing the dependence between two random variables $X$ and $Y$ conditional on a third random vector $Z$ in terms of nonparametric residuals $U_1$ and $U_2$. This paper develops a nonparametric test for conditional independence by combining the partial copula with a quantile regression based method for estimating the nonparametric residuals. We consider a test statistic based on generalized correlation between $U_1$ and $U_2$ and derive its large sample properties under consistency assumptions on the quantile regression procedure. We demonstrate thro…
We reveal the phenomenon that ``naive'' multivariate local polynomial regression can adapt to local smooth lower dimensional structure in the sense that it achieves the optimal convergence rate for nonparametric estimation of regression functions belonging to a Sobolev space when the predictor variables live on or close to a lower dimensional manifold.
This paper proposes a Kolmogorov-Smirnov type statistic and a Cramér-von Mises type statistic to test linearity in semi-functional partially linear regression models. Our test statistics are based on a residual marked empirical process indexed by a randomly projected functional covariate,which is able to circumvent the "curse of dimensionality" brought by the functional covariate. The asymptotic properties of the proposed test statistics under the null, the fixed alternative, and a sequence of local alternatives converging to the null at the $n^{1/2}$ rate are established. A straightforward wi…
A polynomial that is nonnegative over a given interval is called a positive polynomial. The set of such positive polynomials forms a closed convex cone $K$. In this paper, we consider the likelihood ratio test for the hypothesis of positivity that the estimand polynomial regression curve is a positive polynomial. By considering hierarchical hypotheses including the hypothesis of positivity, we define nested likelihood ratio tests, and derive their null distributions as mixtures of chi-square distributions by using the volume-of-tubes method. The mixing probabilities are obtained by utilizing t…
We derive adjusted signed likelihood ratio statistics for a general class of extreme value regression models. The adjustments reduce the error in the standard normal approximation to the distribution of the signed likelihood ratio statistic. We use Monte Carlo simulations to compare the finite-sample performance of the different tests. Our simulations suggest that the signed likelihood ratio test tends to be liberal when the sample size is not large, and that the adjustments are effective in shrinking the size distortion. Two real data applications are presented and discussed.
Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstoc…
We consider the problem of testing significance of predictors in multivariate nonparametric quantile regression. A stochastic process is proposed, which is based on a comparison of the responses with a nonparametric quantile regression estimate under the null hypothesis. It is demonstrated that under the null hypothesis this process converges weakly to a centered Gaussian process and the asymptotic properties of the test under fixed and local alternatives are also discussed. In particular we show, that - in contrast to the nonparametric approach based on estimation of $L^2$-distances - the new…
Model checking plays an important role in linear regression as model misspecification seriously affects the validity and efficiency of regression analysis. In practice, model checking is often performed by subjectively evaluating the plot of the model's residuals. This approach is objectified by constructing a random process from the model's residuals, however due to a very complex covariance function obtaining the exact distribution of the test statistic is intractable. Several solutions to overcome this have been proposed, however the simulation and bootstrap based approaches are only asympt…
Regression is one of the most commonly used statistical techniques. However, testing regression systems is a great challenge because of the absence of test oracle in general. In this paper, we show that Metamorphic Testing is an effective approach to test multiple linear regression systems. In doing so, we identify intrinsic mathematical properties of linear regression, and then propose 11 Metamorphic Relations to be used for testing. Their effectiveness is examined using mutation analysis with a range of different regression programs. We further look at how the testing could be adopted in a m…
Under a standard assumption in complexity theory (NP not in P/poly), we demonstrate a gap between the minimax prediction risk for sparse linear regression that can be achieved by polynomial-time algorithms, and that achieved by optimal algorithms. In particular, when the design matrix is ill-conditioned, the minimax prediction loss achievable by polynomial-time algorithms can be substantially greater than that of an optimal algorithm. This result is the first known gap between polynomial and optimal algorithms for sparse linear regression, and does not depend on conjectures in average-case com…
We use local polynomial fitting to estimate the nonparametric M-regression function for strongly mixing stationary processes $\{(Y_{i},\underline{X}_{i})\}$. We establish a strong uniform consistency rate for the Bahadur representation of estimators of the regression function and its derivatives. These results are fundamental for statistical inference and for applications that involve plugging in such estimators into other functionals where some control over higher order terms are required. We apply our results to the estimation of an additive M-regression model.
In this paper, we study statistical inference in functional quantile regression for scalar response and a functional covariate. Specifically, we consider a functional linear quantile regression model where the effect of the covariate on the quantile of the response is modeled through the inner product between the functional covariate and an unknown smooth regression parameter function that varies with the level of quantile. The objective is to test that the regression parameter is constant across several quantile levels of interest. The parameter function is estimated by combining ideas from f…
Kernel smoothers are considered near the boundary of the interval. Kernels which minimize the expected mean square error are derived. These kernels are equivalent to using a linear weighting function in the local polynomial regression. It is shown that any kernel estimator that satisfies the moment conditions up to order $m$ is equivalent to a local polynomial regression of order $m$ with some non-negative weight function if and only if the kernel has at most $m$ sign changes. A fast algorithm is proposed for computing the kernel estimate in the boundary region for an arbitrary placement of da…
In this paper, we study the detection boundary for minimax hypothesis testing in the context of high-dimensional, sparse binary regression models. Motivated by genetic sequencing association studies for rare variant effects, we investigate the complexity of the hypothesis testing problem when the design matrix is sparse. We observe a new phenomenon in the behavior of detection boundary which does not occur in the case of Gaussian linear regression. We derive the detection boundary as a function of two components: a design matrix sparsity index and signal strength, each of which is a function o…
We propose a goodness-of-fit test for the distribution of errors from a multivariate indirect regression model. The test statistic is based on the Khmaladze transformation of the empirical process of standardized residuals. This goodness-of-fit test is consistent at the root-n rate of convergence, and the test can maintain power against local alternatives converging to the null at a root-n rate.
Gene expression and phenotype association can be affected by potential unmeasured confounders from multiple sources, leading to biased estimates of the associations. Since genetic variants largely explain gene expression variations, they can be used as instruments in studying the association between gene expressions and phenotype in the framework of high dimensional instrumental variable (IV) regression. However, because the dimensions of both genetic variants and gene expressions are often larger than the sample size, statistical inferences such as hypothesis testing for such high dimensional…
This paper presents a goodness-of-fit test for parametric regression models with scalar response and directional predictor, that is, a vector on a sphere of arbitrary dimension. The testing procedure is based on the weighted squared distance between a smooth and a parametric regression estimator, where the smooth regression estimator is obtained by a projected local approach. Asymptotic behavior of the test statistic under the null hypothesis and local alternatives is provided, jointly with a consistent bootstrap algorithm for application in practice. A simulation study illustrates the perform…
This paper studies the high-dimensional mixed linear regression (MLR) where the output variable comes from one of the two linear regression models with an unknown mixing proportion and an unknown covariance structure of the random covariates. Building upon a high-dimensional EM algorithm, we propose an iterative procedure for estimating the two regression vectors and establish their rates of convergence. Based on the iterative estimators, we further construct debiased estimators and establish their asymptotic normality. For individual coordinates, confidence intervals centered at the debiased …
Heteroscedasticity testing is of importance in regression analysis. Existing local smoothing tests suffer severely from curse of dimensionality even when the number of covariates is moderate because of use of nonparametric estimation. In this paper, a dimension reduction-based model adaptive test is proposed which behaves like a local smoothing test as if the number of covariates were equal to the number of their linear combinations in the mean regression function, in particular, equal to 1 when the mean function contains a single index. The test statistic is asymptotically normal under the nu…
This is a comment on Economic Letters DOI http://dx.doi.org/10.1016/j.econlet.2015.10.015. We show that due to some methodological aspects the main conclusions of the above mentioned paper should be a little bit altered.
We propose a kernel machine based hypothesis testing procedure in nonlinear function-on-scalar regression model. Our research is motivated by the Newborn Epigenetic Study (NEST) where the question of interest is whether a pre-specified group of toxic metals or methylation at any of 9 differentially methylated regions (DMRs) is associated with child growth. We take the child growth trajectory as the functional response, and model the toxic metal measurements jointly using a nonlinear function. We use a kernel machine approach to model the unknown function and transform the hypothesis of no effe…
Measuring interdependence between probabilities of default (PDs) in different industry sectors of an economy plays a crucial role in financial stress testing. Thereby, regression approaches may be employed to model the impact of stressed industry sectors as covariates on other response sectors. We identify vine copula based quantile regression as an eligible tool for conducting such stress tests as this method has good robustness properties, takes into account potential nonlinearities of conditional quantile functions and ensures that no quantile crossing effects occur. We illustrate its perfo…
In this paper we propose a new test of heteroscedasticity for parametric regression models and partial linear regression models in high dimensional settings. When the dimension of covariates is large, existing tests of heteroscedasticity perform badly due to the \curse of dimensionality". To attack this problem, we construct a test of heteroscedasticity by using a projection-based empirical process. We study the asymptotic properties of the test statistic under the null hypothesis and alternative hypotheses. It is shown that the test can detect local alternatives departure from the null hypoth…
Testing heteroscedasticity of the errors is a major challenge in high-dimensional regressions where the number of covariates is large compared to the sample size. Traditional procedures such as the White and the Breusch-Pagan tests typically suffer from low sizes and powers. This paper proposes two new test procedures based on standard OLS residuals. Using the theory of random Haar orthogonal matrices, the asymptotic normality of both test statistics is obtained under the null when the degree of freedom tends to infinity. This encompasses both the classical low-dimensional setting where the nu…
Determining a lack of association between an outcome variable and a number of different explanatory variables is frequently necessary in order to disregard a proposed model (i.e., to confirm the lack of an association between an outcome and predictors). Despite this, the literature rarely offers information about, or technical recommendations concerning, the appropriate statistical methodology to be used to accomplish this task. This paper introduces non-inferiority tests for ANOVA and linear regression analyses, that correspond to the standard widely used $F$-test for $\hatη^2$ and $R^{2}$, r…
We develop a maximum-likelihood based method for regression in a setting where the dependent variable is a random graph and covariates are available on a graph-level. The model generalizes the well-known $β$-model for random graphs by replacing the constant model parameters with regression functions. Cramér-Rao bounds are derived for the undirected $β$-model, the directed $β$-model, and the generalized $β$-model. The corresponding maximum likelihood estimators are compared to the bounds by means of simulations. Moreover, examples are given on how to use the presented maximum likelihood estimat…
In the high dimensional regression analysis when the number of predictors is much larger than the sample size, an important question is to select the important variable which are relevant to the response variable of interest. Variable selection and the multiple testing are both tools to address this issue. However, there is little discussion on the connection of these two areas. When the signal strength is strong enough such that the selection consistency is achievable, it seems to be unnecessary to control the false discovery rate. In this paper, we consider the regime where the signals are b…
A formal likelihood ratio hypothesis test for the validity of a parametric regression function is proposed, using a large-dimensional, nonparametric double cone alternative. For example, the test against a constant function uses the alternative of increasing or decreasing regression functions, and the test against a linear function uses the convex or concave alternative. The proposed test is exact, unbiased and the critical value is easily computed. The power of the test increases to one as the sample size increases, under very mild assumptions -- even when the alternative is mis-specified. Th…
This paper deals with the issue of testing hypothesis in symmetric and log-symmetric linear regression models in small and moderate-sized samples. We focus on four tests, namely the Wald, likelihood ratio, score, and gradient tests. These tests rely on asymptotic results and are unreliable when the sample size is not large enough to guarantee a good agreement between the exact distribution of the test statistic and the corresponding chi-squared asymptotic distribution. Bartlett and Bartlett-type corrections typically attenuate the size distortion of the tests. These corrections are available i…
A dimension reduction-based adaptive-to-model test is proposed for significance of a subset of covariates in the context of a nonparametric regression model. Unlike existing local smoothing significance tests, the new test behaves like a local smoothing test as if the number of covariates were just that under the null hypothesis and it can detect local alternatives distinct from the null at the rate that is only related to the number of covariates under the null hypothesis. Thus, the curse of dimensionality is largely alleviated when nonparametric estimation is inevitably required. In the case…
Active learning can reduce the number of samples needed to perform a hypothesis test and to estimate the parameters of a model. In this paper, we revisit the work of Chernoff that described an asymptotically optimal algorithm for performing a hypothesis test. We obtain a novel sample complexity bound for Chernoff's algorithm, with a non-asymptotic term that characterizes its performance at a fixed confidence level. We also develop an extension of Chernoff sampling that can be used to estimate the parameters of a wide variety of models and we obtain a non-asymptotic bound on the estimation erro…
For regression models, most of existing specification tests can be categorized into the class of local smoothing tests and of global smoothing tests. Compared with global smoothing tests, local smoothing tests can only detect local alternatives distinct from the null hypothesis at a much slower rate when the dimension of predictor vector is high, but can be more sensitive to oscillating alternatives. In this paper, we suggest a projection-based test to bridge between the local and global smoothing-based methodologies such that the test can benefit from the advantages of these two types of test…
The semi-parametric Cox proportional hazards regression model has been widely used for many years in several applied sciences. However, a fully parametric proportional hazards model, if appropriately assumed, can often lead to more efficient inference. To tackle the extreme non-robustness of the traditional maximum likelihood estimator in the presence of outliers in the data under such fully parametric proportional hazard models, a robust estimation procedure has recently been proposed extending the concept of the minimum density power divergence estimator (MDPDE) under this set-up. In this pa…
