Inverse probability weighted estimators are the oldest and potentially most commonly used class of procedures for the estimation of causal effects. By adjusting for selection biases via a weighting mechanism, these procedures estimate an effect of interest by constructing a pseudo-population in which selection biases are eliminated. Despite their ease of use, these estimators require the correct specification of a model for the weighting mechanism, are known to be inefficient, and suffer from the curse of dimensionality. We propose a class of nonparametric inverse probability weighted estimators in which the weighting mechanism is estimated via undersmoothing of the highly adaptive lasso, a nonparametric regression function proven to converge at $n^{-1/3}$-rate to the true weighting mechanism. We demonstrate that our estimators are asymptotically linear with variance converging to the nonparametric efficiency bound. Unlike doubly robust estimators, our procedures require neither derivation of the efficient influence function nor specification of the conditional outcome model. Our theoretical developments have broad implications for the construction of efficient inverse probability weighted estimators in large statistical models and a variety of problem settings. We assess the practical performance of our estimators in simulation studies and demonstrate use of our proposed methodology with data from a large-scale epidemiologic study.
In this paper we present new theoretical results for the Dantzig and Lasso estimators of the drift in a high dimensional Ornstein-Uhlenbeck model under sparsity constraints. Our focus is on oracle inequalities for both estimators and error bounds with respect to several norms. In the context of the Lasso estimator our paper is strongly related to [11], who investigated the same problem under row sparsity. We improve their rates and also prove the restricted eigenvalue property solely under ergodicity assumption on the model. Finally, we demonstrate a numerical analysis to uncover the finite sa…
The precision matrix that encodes conditional linear dependency relations among a set of variables forms an important object of interest in multivariate analysis. Sparse estimation procedures for precision matrices such as the graphical lasso (Glasso) gained popularity as they facilitate interpretability, thereby separating pairs of variables that are conditionally dependent from those that are independent (given all other variables). Glasso lacks, however, robustness to outliers. To overcome this problem, one typically applies a robust plug-in procedure where the Glasso is computed from a rob…
Transductive versions of the LASSO and the Dantzig Selector
我们考虑线性回归问题，其中协变量的数量$p$可能大于观测值的数量$n$ $(x_{i},y_{i})_{i\leq i \leq n}$，稀疏假设。一方面，已经成功地提出了几种方法来执行此任务，例如LASSO或Dantzig方法。另一方面，考虑新值$(x_{i})_{n+1\leq i \leq m}$。如果要估计相应的$y_{i}$，应该考虑专门用于此任务的特定估计器，Vapnik称之为“转换”估计器。该估计器可以不同于被设计用于更一般的任务“在整个域上估计”的估计器。在这项工作中，我们提出了一个广义版本的LASSO和Dantzig的，基于几何评论的LASSO在以前的作品。“通常的”LASSO和Dantzig的，以及新的估计解释为LASSO的转导版本，出现作为特殊情况。这些估计是有趣的，至少从理论的角度来看：我们可以给这些估计的假设下，放松版本的假设所需的文件中的“通常”LASSO的理论保证。这些估计也可以有效地计算，与LASSO的结果相媲美。
In this paper we study the properties of the Lasso estimator of the drift component in the diffusion setting. More specifically, we consider a multivariate parametric diffusion model $X$ observed continuously over the interval $[0,T]$ and investigate drift estimation under sparsity constraints. We allow the dimensions of the model and the parameter space to be large. We obtain an oracle inequality for the Lasso estimator and derive an error bound for the $L^2$-distance using concentration inequalities for linear functionals of diffusion processes. The probabilistic part is based upon elements …
We investigate the problem of estimating the drift parameter of a high-dimensional Lévy-driven Ornstein--Uhlenbeck process under sparsity constraints. It is shown that both Lasso and Slope estimators achieve the minimax optimal rate of convergence (up to numerical constants), for tuning parameters chosen independently of the confidence level, which improves the previously obtained results for standard Ornstein--Uhlenbeck processes. The results are nonasymptotic and hold both in probability and conditional expectation with respect to an event resembling the restricted eigenvalue condition.
This paper proposes a theory for $\ell_1$-norm penalized high-dimensional $M$-estimators, with nonconvex risk and unrestricted domain. Under high-level conditions, the estimators are shown to attain the rate of convergence $s_0\sqrt{\log(nd)/n}$, where $s_0$ is the number of nonzero coefficients of the parameter of interest. Sufficient conditions for our main assumptions are then developed and finally used in several examples including robust linear regression, binary classification and nonlinear least squares.
In additive models with many nonparametric components, a number of regularized estimators have been proposed and proven to attain various error bounds under different combinations of sparsity and fixed smoothness conditions. Some of these error bounds match minimax rates in the corresponding settings. Some of the rate minimax methods are non-convex and computationally costly. From these perspectives, the existing solutions to the high-dimensional additive nonparametric regression problem are fragmented. In this paper, we propose a multi-resolution group Lasso (MR-GL) method in a unified approa…
A new estimator, named S-LASSO, is proposed for the coefficient function of the Function-on-Function linear regression model. The S-LASSO estimator is shown to be able to increase the interpretability of the model, by better locating regions where the coefficient function is zero, and to smoothly estimate non-zero values of the coefficient function. The sparsity of the estimator is ensured by a \textit{functional LASSO penalty}, which pointwise shrinks toward zero the coefficient function, while the smoothness is provided by two roughness penalties that penalize the curvature of the final esti…
The purpose of the present paper is to investigate on a class of spherical functional autoregressive processes in order to introduce and study LASSO (Least Absolute Shrinkage and Selection Operator) type estimators for the corresponding autoregressive kernels, defined in the harmonic domain by means of their spectral decompositions. Some crucial properties for these estimators are proved, in particular, consistency and oracle inequalities.
In this paper we introduce and study fused lasso nearly-isotonic signal approximation, which is a combination of fused lasso and generalized nearly-isotonic regression. We show how these three estimators relate to each other, derive solution to the general problem, show that it is computationally feasible and provides a trade-off between piecewise monotonicity, sparsity and goodness-of-fit. Also, we derive an unbiased estimator of the degrees of freedom of the approximator.
University of California, Los Angeles（加州大学洛杉矶分校）；University of Copenhagen（哥本哈根大学）；Aarhus Center for Econometrics (ACE)（奥胡斯计量经济学中心）；Yale University（耶鲁大学）
研究了融合Lasso信号近似器（FLSA）估计加性噪声块状信号序列的性质。我们将FLSA转化为一个普通的Lasso问题。通过研究变换Lasso问题中设计矩阵的性质，我们发现不可表示条件可能不成立，在这种情况下，我们表明FLSA可能无法恢复信号模式。然后，我们将新开发的预处理方法- Puffer变换[Jia and Rohe，2012]应用于变换后的Lasso问题。我们称之为新的方法的预处理融合Lasso，我们给这种方法的非渐近结果。结果表明，当信号跳变强度（相邻两组间的信号差）较大且噪声水平较小时，该预处理融合Lasso估计器能够以较高的概率给出正确的方向图。理论结果给出了什么控制信号模式恢复能力的见解-它是噪声电平{而不是}序列的长度。仿真结果证实了我们的定理，并显示出显着改善的预处理融合Lasso估计香草FLSA。
We consider statistical inference for a single coordinate of regression coefficients in high-dimensional linear models. Recently, the debiased estimators are popularly used for constructing confidence intervals and hypothesis testing in high-dimensional models. However, some representative numerical experiments show that they tend to be biased for large coefficients, especially when the number of large coefficients dominates the number of small coefficients. In this paper, we propose a modified debiased Lasso estimator based on bootstrap. Let us denote the proposed estimator BS-DB for short. W…
Much of the theory for the lasso in the linear model $Y = X β^* + \varepsilon$ hinges on the quantity $2 \| X^\top \varepsilon \|_{\infty} / n$, which we call the lasso's effective noise. Among other things, the effective noise plays an important role in finite-sample bounds for the lasso, the calibration of the lasso's tuning parameter, and inference on the parameter vector $β^*$. In this paper, we develop a bootstrap-based estimator of the quantiles of the effective noise. The estimator is fully data-driven, that is, does not require any additional tuning parameters. We equip our estimator w…
