We consider regression models with parametric (linear or nonlinear) regression function and allow responses to be ``missing at random.'' We assume that the errors have mean zero and are independent of the covariates. In order to estimate expectations of functions of covariate and response we use a fully imputed estimator, namely an empirical estimator based on estimators of conditional expectations given the covariate. We exploit the independence of covariates and errors by writing the conditional expectations as unconditional expectations, which can now be estimated by empirical plug-in estimators. The mean zero constraint on the error distribution is exploited by adding suitable residual-based weights. We prove that the estimator is efficient (in the sense of Hájek and Le Cam) if an efficient estimator of the parameter is used. Our results give rise to new efficient estimators of smooth transformations of expectations. Estimation of the mean response is discussed as a special (degenerate) case.
We consider an empirical likelihood inference for parameters defined by general estimating equations when some components of the random observations are subject to missingness. As the nature of the estimating equations is wide-ranging, we propose a nonparametric imputation of the missing values from a kernel estimator of the conditional distribution of the missing variable given the always observable variable. The empirical likelihood is used to construct a profile likelihood for the parameter of interest. We demonstrate that the proposed nonparametric imputation can remove the selection bias …
In a seminal paper \cite{studden1968} characterized $c$-optimal designs in regression models, where the regression functions form a Chebyshev system. He used these results to determine the optimal design for estimating the individual coefficients in a polynomial regression model on the interval $[-1,1]$ explicitly. In this note we identify the optimal design for estimating the individual coefficients in a polynomial regression model with no intercept (here the regression functions do not form a Chebyshev system).
This paper proposes a hierarchical method for estimating the location parameters of a multivariate vector in the presence of missing data. At i th step of this procedure an estimate of the location parameters for non-missing components of the vector is based on combining the information in the subset of observations with the non-missing components with updated estimates of the location parameters from all subsets with even more missing components in an iterative fashion. If the variance-covariance matrix is known, then the resulting estimator is unbiased with the smallest variance provided mis…
This paper proposes a general family of estimators for estimating the population mean in systematic sampling in the presence of non-response adapting the family of estimators proposed by Khoshnevisan et al. (2007). In this paper we have discussed the general properties of the proposed family including optimum property. The results have been illustrated numerically by taking an empirical population considered in the literature.
We construct an efficient estimator for the error distribution function of the nonparametric regression model Y = r(Z) + e. Our estimator is a kernel smoothed empirical distribution function based on residuals from an under-smoothed local quadratic smoother for the regression function.
In this note we consider the optimal design problem for estimating the slope of a polynomial regression with no intercept at a given point, say z. In contrast to previous work, which considers symmetric design spaces we investigate the model on the interval $[0, a]$ and characterize those values of $z$, where an explicit solution of the optimal design is possible.
This paper investigates the large sample properties of the variance, weights, and risk of high-dimensional portfolios where the inverse of the covariance matrix of excess asset returns is estimated using a technique called nodewise regression. Nodewise regression provides a direct estimator for the inverse covariance matrix using the Least Absolute Shrinkage and Selection Operator (Lasso) of Tibshirani (1994) to estimate the entries of a sparse precision matrix. We show that the variance, weights, and risk of the global minimum variance portfolios and the Markowitz mean-variance portfolios are…
Non linear regression models are a standard tool for modeling real phenomena, with several applications in machine learning, ecology, econometry... Estimating the parameters of the model has garnered a lot of attention during many years. We focus here on a recursive method for estimating parameters of non linear regressions. Indeed, these kinds of methods, whose most famous are probably the stochastic gradient algorithm and its averaged version, enable to deal efficiently with massive data arriving sequentially. Nevertheless, they can be, in practice, very sensitive to the case where the eigen…
We derive information bounds for the regression parameters in Cox models when data are missing at random. These calculations are of interest for understanding the behavior of efficient estimation in case-cohort designs, a type of two-phase design often used in cohort studies. The derivations make use of key lemmas appearing in Robins, Rotnitzky and Zhao [J. Amer. Statist. Assoc. 89 (1994) 846-866] and Robins, Hsieh and Newey [J. Roy. Statist. Soc. Ser. B 57 (1995) 409-424], but in a form suited for our purposes here. We begin by summarizing the results of Robins, Rotnitzky and Zhao in a form t…
We study the asymptotic behavior of a class of methods for sufficient dimension reduction in high-dimension regressions, as the sample size and number of predictors grow in various alignments. It is demonstrated that these methods are consistent in a variety of settings, particularly in abundant regressions where most predictors contribute some information on the response, and oracle rates are possible. Simulation results are presented to support the theoretical conclusion.
We consider a two-stage procedure (TSP) for estimating an inverse regression function at a given point, where isotonic regression is used at stage one to obtain an initial estimate and a local linear approximation in the vicinity of this estimate is used at stage two. We establish that the convergence rate of the second-stage estimate can attain the parametric $n^{1/2}$ rate. Furthermore, a bootstrapped variant of TSP (BTSP) is introduced and its consistency properties studied. This variant manages to overcome the slow speed of the convergence in distribution and the estimation of the derivati…
In this paper we present the practical benefits of a new random forest algorithm to deal withmissing values in the sample. The purpose of this work is to compare the different solutionsto deal with missing values with random forests and describe our new algorithm performanceas well as its algorithmic complexity. A variety of missing value mechanisms (such as MCAR,MAR, MNAR) are considered and simulated. We study the quadratic errors and the bias ofour algorithm and compare it to the most popular missing values random forests algorithms inthe literature. In particular, we compare those techniqu…
We consider the problem of consistently estimating the conditional distribution $P(Y \in A |X)$ of a functional data object $Y=(Y(t): t\in[0,1])$ given covariates $X$ in a general space, assuming that $Y$ and $X$ are related by a functional linear regression model. Two natural estimation methods are proposed, based on either bootstrapping the estimated model residuals, or fitting functional parametric models to the model residuals and estimating $P(Y \in A |X)$ via simulation. Whether either of these methods lead to consistent estimation depends on the consistency properties of the regression …
This paper considers an estimation of semiparametric functional (varying)-coefficient quantile regression with spatial data. A general robust framework is developed that treats quantile regression for spatial data in a natural semiparametric way. The local M-estimators of the unknown functional-coefficient functions are proposed by using local linear approximation, and their asymptotic distributions are then established under weak spatial mixing conditions allowing the data processes to be either stationary or nonstationary with spatial trends. Application to a soil data set is demonstrated wi…
We study the problem of estimating piecewise monotone vectors. This problem can be seen as a generalization of the isotonic regression that allows a small number of order-violating changepoints. We focus mainly on the performance of the nearly-isotonic regression proposed by Tibshirani et al. (2011). We derive risk bounds for the nearly-isotonic regression estimators that are adaptive to piecewise monotone signals. The estimator achieves a near minimax convergence rate over certain classes of piecewise monotone signals under a weak assumption. Furthermore, we present an algorithm that can be a…
Although the interest in the the use of social and information networks has grown, most inferences on networks assume the data collected represents the complete. However, when ignoring missing data, even when missing completely at random, this results in bias for estimators regarding inference network related parameters. In this paper, we focus on constructing estimators for the probability that a randomly selected node has node has at least one edge under the assumption that nodes are missing completely at random along with their corresponding edges. In addition, issues also arise in obtainin…
The paper considers the problem of estimating a $p\geq2$\ dimensional mean vector of a multivariate conditionally normal distribution under quadratic loss. The problem of this type arises when estimating the parameters in a continuous time regression model with a non-Gaussian Ornstein--Uhlenbeck process driven by the mixture of a Brownian motion and a compound Poisson process.
In this paper we consider a regression model that allows for time series covariates as well as heteroscedasticity with a regression function that is modelled nonparametrically. We assume that the regression function changes at some unknown time $\lfloor ns_0\rfloor$, $s_0\in(0,1)$, and our aim is to estimate the (rescaled) change point $s_0$. The considered estimator is based on a Kolmogorov-Smirnov functional of the marked empirical process of residuals. We show consistency of the estimator and prove a rate of convergence of $O_P(n^{-1})$ which in this case is clearly optimal as there are onl…
This article introduces a new nonparametric method for estimating a univariate regression function of bounded variation. The method exploits the Jordan decomposition which states that a function of bounded variation can be decomposed as the sum of a non-decreasing function and a non-increasing function. This suggests combining the backfitting algorithm for estimating additive functions with isotonic regression for estimating monotone functions. The resulting iterative algorithm is called Iterative Isotonic Regression (I.I.R.). The main technical result in this paper is the consistency of the p…
kNN estimation in semi-functional partial linear regression with missing responses at random
Universidade da Coruña（拉科鲁尼亚大学）；Centro de Investigación en Tecnologías de la Información y las Comunicaciones (CITIC)（信息与通信技术研究中心）；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）
Complete randomization allows for consistent estimation of the average treatment effect based on the difference in means of the outcomes without strong modeling assumptions on the outcome-generating process. Appropriate use of the pretreatment covariates can further improve the estimation efficiency. However, missingness in covariates is common in experiments and raises an important question: should we adjust for covariates subject to missingness, and if so, how? The unadjusted difference in means is always unbiased. The complete-covariate analysis adjusts for all completely observed covariate…
A sample covariance matrix $\boldsymbol{S}$ of completely observed data is the key statistic in a large variety of multivariate statistical procedures, such as structured covariance/precision matrix estimation, principal component analysis, and testing of equality of mean vectors. However, when the data are partially observed, the sample covariance matrix from the available data is biased and does not provide valid multivariate procedures. To correct the bias, a simple adjustment method called inverse probability weighting (IPW) has been used in previous research, yielding the IPW estimator. T…
This paper considers the problem of kernel regression and classification with possibly unobservable response variables in the data, where the mechanism that causes the absence of information is unknown and can depend on both predictors and the response variables. Our proposed approach involves two steps: In the first step, we construct a family of models (possibly infinite dimensional) indexed by the unknown parameter of the missing probability mechanism. In the second step, a search is carried out to find the empirically optimal member of an appropriate cover (or subclass) of the underlying f…
In this paper, we are interested in nonparametric kernel estimation of a generalized regression function, including conditional cumulative distribution and conditional quantile functions, based on an incomplete sample $(X_t, Y_t, ζ_t)_{t\in \mathbb{ R}^+}$ copies of a continuous-time stationary ergodic process $(X, Y, ζ)$. The predictor $X$ is valued in some infinite-dimensional space, whereas the real-valued process $Y$ is observed when $ζ= 1$ and missing whenever $ζ= 0$. Pointwise and uniform consistency (with rates) of these estimators as well as a central limit theorem are established. Con…
Depth notions in regression have been systematically proposed and examined in Zuo (2018). One of the prominent advantages of notion of depth is that it can be directly utilized to introduce median-type deepest estimating functionals (or estimators in empirical distribution case) for location or regression parameters in a multi-dimensional setting. Regression depth shares the advantage. Depth induced deepest estimating functionals are expected to inherit desirable and inherent robustness properties ( e.g. bounded maximum bias and influence function and high breakdown point) as their univariate …
Georgia State University（佐治亚州立大学）；University of Waterloo（滑铁卢大学）
We present a robust framework to perform linear regression with missing entries in the features. By considering an elliptical data distribution, and specifically a multivariate normal model, we are able to conditionally formulate a distribution for the missing entries and present a robust framework, which minimizes the worst case error caused by the uncertainty about the missing data. We show that the proposed formulation, which naturally takes into account the dependency between different variables, ultimately reduces to a convex program, for which a customized and scalable solver can be deli…
Augmented transfer regression learning for completely missing covariates
School of Mathematics and Statistics, Huazhong University of Science and Technology（华中科技大学数学与统计学院）；Department of Statistics, Colorado State University（科罗拉多州立大学统计学系）
The development of coherent missing data models to account for nonmonotone missing at random (MAR) data by inverse probability weighting (IPW) remains to date largely unresolved. As a consequence, IPW has essentially been restricted for use only in monotone missing data settings. We propose a class of models for nonmonotone missing data mechanisms that spans the MAR model, while allowing the underlying full data law to remain unrestricted. For parametric specifications within the proposed class, we introduce an unconstrained maximum likelihood estimator for estimating the missing data probabil…
Doubly robust target inference for generalized linear regression with completely missing covariates
School of Mathematics and Statistics, Huazhong University of Science and Technology（华中科技大学数学与统计学院）；Department of Statistics and Data Science, Tsinghua University（清华大学统计与数据科学系）
We study the problem of estimating a functional or a parameter in the context where outcome is subject to nonignorable missingness. We completely avoid modeling the regression relation, while allowing the propensity to be modeled by a semiparametric logistic relation where the dependence on covariates is unspecified. We discover a surprising phenomenon in that the estimation of the parameter in the propensity model as well as the functional estimation can be carried out without assessing the missingness dependence on covariates. This allows us to propose a general class of estimators for both …
I propose kernel ridge regression estimators for nonparametric dose response curves and semiparametric treatment effects in the setting where an analyst has access to a selected sample rather than a random sample; only for select observations, the outcome is observed. I assume selection is as good as random conditional on treatment and a sufficiently rich set of observed covariates, where the covariates are allowed to cause treatment or be caused by treatment -- an extension of missingness-at-random (MAR). I propose estimators of means, increments, and distributions of counterfactual outcomes …
The moment conditions or estimating equations for instrumental variables quantile regression involve the discontinuous indicator function. We instead use smoothed estimating equations (SEE), with bandwidth $h$. We show that the mean squared error (MSE) of the vector of the SEE is minimized for some $h>0$, leading to smaller asymptotic MSE of the estimating equations and associated parameter estimators. The same MSE-optimal $h$ also minimizes the higher-order type I error of a SEE-based $χ^2$ test and increases size-adjusted power in large samples. Computation of the SEE estimator also becomes …
Although the standard formulations of prediction problems involve fully-observed and noiseless data drawn in an i.i.d. manner, many applications involve noisy and/or missing data, possibly involving dependence, as well. We study these issues in the context of high-dimensional sparse linear regression, and propose novel estimators for the cases of noisy, missing and/or dependent data. Many standard approaches to noisy or missing data, such as those using the EM algorithm, lead to optimization problems that are inherently nonconvex, and it is difficult to establish theoretical guarantees on prac…
We revisit the problem of \emph{missing mass concentration}, developing a new method of estimating concentration of heterogenic sums, in spirit of celebrated Rosenthal's inequality. As a result we slightly improve the state-of-art bounds due to Ben-Hamou at al., and simplify the proofs.
