arXiv 1002.15832010-02-11math.STstat.TH

用于高维变量选择和统计估计的双线性Lasso

Thresholded Lasso for high dimensional variable selection and statistical estimation

Shuheng Zhou

备注：: 37 Pages, 4 Figures; Parts of the paper have appeared in Proceedings of Advances in Neural Information Processing Systems 22, December, 2009 (NIPS 2009)

摘要：

给定$n$噪声样本与$p$维度，其中$n \ll p$，我们表明，基于Lasso的多步阈值处理过程-我们称之为{ \it holded Lasso}，可以准确地估计线性模型$Y = X β+ ε$中的稀疏向量$β\in \R^p$，其中$X_{n \times p}$是一个归一化为列$\ell_2$范数$\sqrt{n}$和$ε\sim N(0, σ^2 I_n)$的设计矩阵。我们表明，在限制特征值（RE）条件下，（Bickel-Ritov-Tsybakov 09），在选择足够稀疏的模型时，可以在理想均方误差的对数因子内实现$\ell_2$损失，从而实现{ \it稀疏预言不等式};预言机将提供关于哪些坐标是非零的以及哪些坐标在噪声水平之上的完美信息。从某种意义上说，保留的Lasso恢复了$\ell_0$惩罚最小二乘估计量所做的选择，因为它选择了一个足够稀疏的模型，而不会牺牲估计$β$和预测$X β$的准确性。我们还证明了高斯-丹齐格选择器（Candès-Tao 07），如果$X$服从一致不确定性原则，并且如果真参数足够稀疏，则将实现如上所述的稀疏预言不等式，同时在最坏情况下允许模型中最多有$s_0$个无关变量，其中$s_0 \leq s$是最小整数，使得对于$λ= \sqrt{2 \log p/n}$，$\sum_{i=1}^p \min(β_i^2, λ^2 σ^2) \leq s_0 λ^2 σ^2$。我们的模拟结果与我们的理论分析吻合得很好。

英文摘要：

Given $n$ noisy samples with $p$ dimensions, where $n \ll p$, we show that the multi-step thresholding procedure based on the Lasso -- we call it the {\it Thresholded Lasso}, can accurately estimate a sparse vector $β\in \R^p$ in a linear model $Y = X β+ ε$, where $X_{n \times p}$ is a design matrix normalized to have column $\ell_2$ norm $\sqrt{n}$, and $ε\sim N(0, σ^2 I_n)$. We show that under the restricted eigenvalue (RE) condition (Bickel-Ritov-Tsybakov 09), it is possible to achieve the $\ell_2$ loss within a logarithmic factor of the ideal mean square error one would achieve with an {\em oracle} while selecting a sufficiently sparse model -- hence achieving {\it sparse oracle inequalities}; the oracle would supply perfect information about which coordinates are non-zero and which are above the noise level. In some sense, the Thresholded Lasso recovers the choices that would have been made by the $\ell_0$ penalized least squares estimators, in that it selects a sufficiently sparse model without sacrificing the accuracy in estimating $β$ and in predicting $X β$. We also show for the Gauss-Dantzig selector (Candès-Tao 07), if $X$ obeys a uniform uncertainty principle and if the true parameter is sufficiently sparse, one will achieve the sparse oracle inequalities as above, while allowing at most $s_0$ irrelevant variables in the model in the worst case, where $s_0 \leq s$ is the smallest integer such that for $λ= \sqrt{2 \log p/n}$, $\sum_{i=1}^p \min(β_i^2, λ^2 σ^2) \leq s_0 λ^2 σ^2$. Our simulation results on the Thresholded Lasso match our theoretical analysis excellently.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文