Yongwu Shao, R Dennis Cook, Sanford Weisberg
We show that under a linearity condition on the distribution of the predictors, the coefficient in single-index regression can be estimated with the same efficiency as in the case when the link function is known. Thus, the linearity condition seems to substitute for knowing the exact conditional distribution of the response given the linear combinations of the predictors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
We consider a regression model with errors that are a.s. negative. Thus the regression function is not the expected value of the observations but the right endpoint of their support. We develop two goodness-of-fit tests for the hypotheses that the regression function is an affine function, study the asymptotic distributions of the test statistics in order to approximately fix the sizes of the tests, derive their finite-sample properties based on simulations and apply them to life expectancy data.
This paper proposes a Kolmogorov-Smirnov type statistic and a Cramér-von Mises type statistic to test linearity in semi-functional partially linear regression models. Our test statistics are based on a residual marked empirical process indexed by a randomly projected functional covariate,which is able to circumvent the "curse of dimensionality" brought by the functional covariate. The asymptotic properties of the proposed test statistics under the null, the fixed alternative, and a sequence of local alternatives converging to the null at the $n^{1/2}$ rate are established. A straightforward wi…
Single-index models for extreme value index regression
针对协变量高维时EVI非参数回归的维数灾难问题，在帕累托型尾分布下将单指标模型应用于EVI回归，提出惩罚极大似然估计并推导其渐近性质，数值研究验证了模型有效性。
The single-index model is a statistical model for intrinsic regression where responses are assumed to depend on a single yet unknown linear combination of the predictors, allowing to express the regression function as $ \mathbb{E} [ Y | X ] = f ( \langle v , X \rangle ) $ for some unknown \emph{index} vector $v$ and \emph{link} function $f$. Conditional methods provide a simple and effective approach to estimate $v$ by averaging moments of $X$ conditioned on $Y$, but depend on parameters whose optimal choice is unknown and do not provide generalization bounds on $f$. In this paper we propose a…
Hypothesis testing for varying coefficient models in tail index regression
针对尾指数回归缺乏变系数模型检验方法的问题，本文研究估计量的渐近性质，提出用于判断系数函数是否显著随协变量变化的假设检验方法。
In this article, we propose some new generalizations of M-estimation procedures for single-index regression models in presence of randomly right-censored responses. We derive consistency and asymptotic normality of our estimates. The results are proved in order to be adapted to a wide range of techniques used in a censored regression framework (e.g. synthetic data or weighted least squares). As in the uncensored case, the estimator of the single-index parameter is seen to have the same asymptotic behavior as in a fully parametric scheme. We compare these new estimators with those based on the …
We propose a class of weighted least squares estimators for the tail index of a distribution function with a regularly varying upper tail. Our approach is based on the method developed by \cite{Holan2010} for the Parzen tail index. Asymptotic normality of the estimators is proved. Through a simulation study, these and earlier estimators are compared in the Pareto and Hall models using the mean squared error as criterion. The results show that the weighted least squares estimator is better than the other estimators investigated.
Under a single-index regression assumption, we introduce a new semiparametric procedure to estimate a conditional density of a censored response. The regression model can be seen as a generalization of Cox regression model and also as a profitable tool to perform dimension reduction under censoring. This technique extends the results of Delecroix et al. (2003). We derive consistency and asymptotic normality of our estimator of the index parameter by proving its asymptotic equivalence with the (uncomputable) maximum likelihood estimator, using martingales results for counting processes and argu…
Statistical learning evolves quickly with more and more sophisticated models proposed to incorporate the complicated data structure from modern scientific and business problems. Varying index coefficient models extend varying coefficient models and single index models, becoming the latest state-of-the-art for semiparametric regression. This new class of models offers greater flexibility to characterize complicated nonlinear interaction effects in regression analysis. To safeguard against outliers and extreme observations, we consider a robust quantile regression approach to estimate the model …
We propose a two-step pseudo-maximum likelihood procedure for semiparametric single-index regression models where the conditional variance is a known function of the regression and an additional parameter. The Poisson single-index regression with multiplicative unobserved heterogeneity is an example of such models. Our procedure is based on linear exponential densities with nuisance parameter. The pseudo-likelihood criterion we use contains a nonparametric estimate of the index regression and therefore a rule for choosing the smoothing parameter is needed. We propose an automatic and natural r…
Stochastic simulators such as Monte-Carlo estimators are widely used in science and engineering to study physical systems through their probabilistic representation. Global sensitivity analysis aims to identify the input parameters which have the most important impact on the output. A popular tool to perform global sensitivity analysis is the variance-based method which comes from the Hoeffding-Sobol decomposition. Nevertheless, this method requires an important number of simulations and is often unfeasible under reasonable time constraint. Therefore, an approximation of the input/output relat…
Gaussian approximation for non-linearity parameter estimation in perturbed random fields on the sphere
研究球面上受扰随机场非线性参数估计，通过将估计问题转化为普通最小二乘回归得出渐近矩，结合斯坦 - 马利瓦因技术得到定量高斯近似及收敛速率，数值实验验证相关规律和行为。
Consider a random vector (X',Y)', where X is d-dimensional and Y is one-dimensional. We assume that Y is subject to random right censoring. The aim of this paper is twofold. First, we propose a new estimator of the joint distribution of (X',Y)'. This estimator overcomes the common curse-of-dimensionality problem, by using a new dimension reduction technique. Second, we assume that the relation between X and Y is given by a mean regression single index model, and propose a new estimator of the parameters in this model. The asymptotic properties of all proposed estimators are obtained.
In this paper, based on the kernel estimator proposed by Ould-Said and Lemdani (Ann. Instit. Statist. Math. 2006), we develop some new generalized M-estimator procedures for single index regression models with left-truncated responses. The consistency and asymptotic normality of our estimators are also established. Some simulation studies are given to investigate the finite sample performance of the proposed estimators.
The multi-index model is a simple yet powerful high-dimensional regression model which circumvents the curse of dimensionality assuming $ \mathbb{E} [ Y | X ] = g(A^\top X) $ for some unknown index space $A$ and link function $g$. In this paper we introduce a method for the estimation of the index space, and study the propagation error of an index space estimate in the regression of the link function. The proposed method approximates the index space by the span of linear regression slope coefficients computed over level sets of the data. Being based on ordinary least squares, our approach is e…
This paper discusses asymptotically distribution free tests for the classical goodness-of-fit hypothesis of an error distribution in nonparametric regression models. These tests are based on the same martingale transform of the residual empirical process as used in the one sample location model. This transformation eliminates extra randomization due to covariates but not due the errors, which is intrinsically present in the estimators of the regression function. Thus, tests based on the transformed process have, generally, better power. The results of this paper are applicable as soon as asymp…
In this article, we consider the problem of estimating the index parameter $α_0$ in the single index model $E[Y |X] = f_0(α_0^T X)$ with $f_0$ the unknown ridge function defined on $\mathbb{R}$, $X$ a d-dimensional covariate and $Y$ the response. We show that when $X$ is Gaussian, then $α_0$ can be consistently estimated by regressing the observed responses $Y_i$, $i = 1, . . ., n$ on the covariates $X_1, . . ., X_n$ after centering and rescaling. The method works without any additional smoothness assumptions on $f_0$ and only requires that $cov(f_0(α_0^T X),α_0^TX) \neq 0$, which is always sa…
Estimating the tail index parameter is one of the primal objectives in extreme value theory. For heavy-tailed distributions the Hill estimator is the most popular way to estimate the tail index parameter. Improving the Hill estimator was aimed by recent works with different methods, for example by using bootstrap, or Kolmogorov-Smirnov metric. These methods are asymptotically consistent, but for tail index $ξ>1$ and smaller sample sizes the estimation fails to approach the theoretical value for realistic sample sizes. In this paper, we introduce a new empirical method, which can estimate high …
Single Index Fréchet Regression
本文提出单指标 Fréchet 回归，用投影方向参数弥补 Fréchet 回归缺乏线性参数的不足，推导参数估计的渐近分布以支持假设检验，并通过模拟和 ADNI 静息态 fMRI 数据验证方法。
In many applications, the dataset under investigation exhibits heterogeneous regimes that are more appropriately modeled using piece-wise linear models for each of the data segments separated by change-points. Although there have been much work on change point linear regression for the low dimensional case, high-dimensional change point regression is severely underdeveloped. Motivated by the analysis of Minnesota House Price Index data, we propose a fully Bayesian framework for fitting changing linear regression models in high-dimensional settings. Using segment-specific shrinkage and diffusio…
In a multi-index model with $k$ index vectors, the input variables are transformed by taking inner products with the index vectors. A transfer function $f: \mathbb{R}^k \to \mathbb{R}$ is applied to these inner products to generate the output. Thus, multi-index models are a generalization of linear models. In this paper, we consider monotone multi-index models. Namely, the transfer function is assumed to be coordinate-wise monotone. The monotone multi-index model therefore generalizes both linear regression and isotonic regression, which is the estimation of a coordinate-wise monotone function…
A Two-Step Projection-Based Goodness-of-Fit Test for Ultra-High Dimensional Sparse Regressions
本文提出了一种针对超高维稀疏回归的两步投影型适配性检验方法，通过构建投影统计量并结合p值组合程序，提高了在高维数据下的检验功效。
Fully nonparametric methods for regression from functional data have poor accuracy from a statistical viewpoint, reflecting the fact that their convergence rates are slower than nonparametric rates for the estimation of high-dimensional functions. This difficulty has led to an emphasis on the so-called functional linear model, which is much more flexible than common linear models in finite dimension, but nevertheless imposes structural constraints on the relationship between predictors and responses. Recent advances have extended the linear approach by using it in conjunction with link functio…
The problem of adaptive multivariate function estimation in the single-index regression model with random design and weak assumptions on the noise is investigated. A novel estimation procedure that adapts simultaneously to the unknown index vector and the smoothness of the link function by selecting from a family of specific kernel estimators is proposed. We establish a pointwise oracle inequality which, in its turn, is used to judge the quality of estimating the entire function (``global'' oracle inequality). Both the results are applied to the problems of pointwise and global adaptive estima…
Automatic and location-adaptive estimation in functional single-index regression
针对函数型单指标模型的回归估计问题，提出基于k近邻的自动位置自适应方法，实现数据驱动选参，预测性能优于核估计，并补充了相关核估计的一致渐近结果。
We study a regression problem where for some part of the data we observe both the label variable ($Y$) and the predictors (${\bf X}$), while for other part of the data only the predictors are given. Such a problem arises, for example, when observations of the label variable are costly and may require a skilled human agent. When the conditional expectation $E[Y | {\bf X}]$ is not exactly linear, one can consider the best linear approximation to the conditional expectation, which can be estimated consistently by the least squares estimates (LSE). The latter depends only on the labeled data. We s…
Conditional Variance Estimation (CVE) is a novel sufficient dimension reduction (SDR) method for additive error regressions with continuous predictors and link function. It operates under the assumption that the predictors can be replaced by a lower dimensional projection without loss of information. In contrast to the majority of moment based sufficient dimension reduction methods, Conditional Variance Estimation is fully data driven, does not require the restrictive linearity and constant variance conditions, and is not based on inverse regression. CVE is shown to be consistent and its objec…
This paper studies a class of exponential family models whose canonical parameters are specified as linear functionals of an unknown infinite-dimensional slope function. The optimal minimax rates of convergence for slope function estimation are established. The estimators that achieve the optimal rates are constructed by constrained maximum likelihood estimation with parameters whose dimension grows with sample size. A change-of-measure argument, inspired by Le Cam's theory of asymptotic equivalence, is used to eliminate the bias caused by the non-linearity of exponential family models.
For the past two decades, single-index model, a special case of projection pursuit regression, has proven to be an efficient way of coping with the high dimensional problem in nonparametric regression. In this paper, based on weakly dependent sample, we investigate the single-index prediction (SIP) model which is robust against deviation from the single-index model. The single-index is identified by the best approximation to the multivariate prediction function of the response variable, regardless of whether the prediction function is a genuine single-index function. A polynomial spline estima…
The central subspace of a pair of random variables $(y,x) \in \mathbb{R}^{p+1}$ is the minimal subspace $\mathcal{S}$ such that $y \perp \hspace{-2mm} \perp x\mid P_{\mathcal{S}}x$. In this paper, we consider the minimax rate of estimating the central space of the multiple index models $y=f(β_{1}^τx,β_{2}^τx,...,β_{d}^τx,ε)$ with at most $s$ active predictors where $x \sim N(0,I_{p})$. We first introduce a large class of models depending on the smallest non-zero eigenvalue $λ$ of $var(\mathbb{E}[x|y])$, over which we show that an aggregated estimator based on the SIR procedure converges at rat…
We propose a method for average partial effect estimation in high-dimensional single-index models that is root-n-consistent and asymptotically unbiased given sparsity assumptions on the underlying regression model. This note was prepared as a comment on Wooldridge and Zhu [2018], forthcoming in the Journal of Business and Economic Statistics.
In partially linear single-index models, we obtain the semiparametrically efficient profile least-squares estimators of regression coefficients. We also employ the smoothly clipped absolute deviation penalty (SCAD) approach to simultaneously select variables and estimate regression coefficients. We show that the resulting SCAD estimators are consistent and possess the oracle property. Subsequently, we demonstrate that a proposed tuning parameter selector, BIC, identifies the true model consistently. Finally, we develop a linear hypothesis test for the parametric coefficients and a goodness-of-…
The problem of statistical inference for regression coefficients in a high-dimensional single-index model is considered. Under elliptical symmetry, the single index model can be reformulated as a proxy linear model whose regression parameter is identifiable. We construct estimates of the regression coefficients of interest that are similar to the debiased lasso estimates in the standard linear model and exhibit similar properties: root-n-consistency and asymptotic normality. The procedure completely bypasses the estimation of the unknown link function, which can be extremely challenging depend…
Self-convolved Bootstrap for M-regression under Complex Temporal Dynamics
针对协变量与误差可交叉依赖的非平稳时变系数 M 回归，提出单参数自卷积 bootstrap，实现 √n 一致推断及多类统一检验，并应用于气候与股票数据。
Finite Sample Confidence Regions for Linear Regression Parameters Using Arbitrary Predictors
本研究提出一种利用任意预测器构建线性回归参数有限样本置信区域的方法，该方法对噪声假设少、适用函数范围广，可转化为混合整数线性规划约束实现优化与区间提取，置信区域为空的特性可用于假设检验，且已在合成数据上验证了适用性。
This paper studies estimation in functional linear quantile regression in which the dependent variable is scalar while the covariate is a function, and the conditional quantile for each fixed quantile index is modeled as a linear functional of the covariate. Here we suppose that covariates are discretely observed and sampling points may differ across subjects, where the number of measurements per subject increases as the sample size. Also, we allow the quantile index to vary over a given subset of the open unit interval, so the slope function is a function of two variables: (typically) time an…
A model-assisted semiparametric method of estimating finite population totals is investigated to improve the precision of survey estimators by incorporating multivariate auxiliary information. The proposed superpopulation model is a single-index model which has proven to be a simple and efficient semiparametric tool in multivariate regression. A class of estimators based on polynomial spline regression is proposed. These estimators are robust against deviation from single-index models. Under standard design conditions, the proposed estimators are asymptotically design-unbiased, consistent and …
We propose three novel consistent specification tests for quantile regression models which generalize former tests in three ways. First, we allow the covariate effects to be quantile-dependent and nonlinear. Second, we allow parameterizing the conditional quantile functions by appropriate basis functions, rather than parametrically. We are hence able to test for functional forms beyond linearity, while retaining the linear effects as special cases. In both cases, the induced class of conditional distribution functions is tested with a Cramér-von Mises type test statistic for which we derive th…
In this article we investigate consistency of selection in regression models via the popular Lasso method. Here we depart from the traditional linear regression assumption and consider approximations of the regression function $f$ with elements of a given dictionary of $M$ functions. The target for consistency is the index set of those functions from this dictionary that realize the most parsimonious approximation to $f$ among all linear combinations belonging to an $L_2$ ball centered at $f$ and of radius $r_{n,M}^2$. In this framework we show that a consistent estimate of this index set can …
It has previously been shown that ordinary least squares can be used to estimate the coefficients of the single-index model under only mild conditions. However, the estimator is non-robust leading to poor estimates for some models. In this paper we propose a new sliced least-squares estimator that utilizes ideas from Sliced Inverse Regression. Slices with problematic observations that contribute to high variability in the estimator can easily be down-weighted to robustify the procedure. The estimator is simple to implement and can result in vast improvements for some models when compared to th…
Single index model is a powerful yet simple model, widely used in statistics, machine learning, and other scientific fields. It models the regression function as $g(<a,x>)$, where a is an unknown index vector and x are the features. This paper deals with a nonlinear generalization of this framework to allow for a regressor that uses multiple index vectors, adapting to local changes in the responses. To do so we exploit the conditional distribution over function-driven partitions, and use linear regression to locally estimate index vectors. We then regress by applying a kNN type estimator that …
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。