arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1310.1969stat.MEmath.STstat.TH
收藏

通过排序L1范数的统计估计和检验

Statistical estimation and testing via the sorted L1 norm

Malgorzata Bogdan, Ewout van den Berg, Weijie Su, Emmanuel Candes

摘要:

我们介绍了一种新的方法,稀疏回归和变量选择,这是现代思想的启发,在多重检验。假设我们有来自线性模型y = X beta + z的观测值,那么我们建议通过称为SLOPE的新估计量来估计回归系数,这是最小化0.5的解决方案||y - Xb\|_2^2 + Xbda_1 |b| _(1)+Xbda_2 |b| _(2)+. + lambda_p |b| _(p);这里,λ 1>= λ 2>=. >= λ_p >= 0和|b| _(1)>= |b| _(2)>=. >= |b| _(p)是b的幅度的顺序统计量。正则化子是一个排序的L1范数,它根据回归系数的秩对它们进行惩罚:秩越高,惩罚越大。这类似于著名的BHq程序[Benjamini和Hochberg,1995],该程序将从一个家族中获取的检验统计量的值与取决于其在家族中的排名的临界阈值进行比较。SLOPE是一个凸规划,我们证明了一个有效的算法计算的解决方案。本文证明了对于p个变量的正交设计,取lambda_i = F^{-1}(1-q_i)(F是误差的cdf),q_i = iq/(2 p),控制变量选择的错误发现率(FDR)。当设计矩阵为非正交时,FDR水平和功率存在固有限制,这可以通过基于L1-like罚的模型选择方法获得。然而,每当设计矩阵的列不是强相关的,我们凭经验证明,只要非零系数的数量不是太大,就有可能选择参数λ_i,以获得在合理的水平上的FDR控制。与此同时,该过程表现出比套索更大的功率,套索平等地对待所有系数。本文通过全面的模拟研究进一步说明了新的选择规则的估计性能。

英文摘要:

We introduce a novel method for sparse regression and variable selection, which is inspired by modern ideas in multiple testing. Imagine we have observations from the linear model y = X beta + z, then we suggest estimating the regression coefficients by means of a new estimator called SLOPE, which is the solution to minimize 0.5 ||y - Xb\|_2^2 + lambda_1 |b|_(1) + lambda_2 |b|_(2) + ... + lambda_p |b|_(p); here, lambda_1 >= λ_2 >= ... >= λ_p >= 0 and |b|_(1) >= |b|_(2) >= ... >= |b|_(p) is the order statistic of the magnitudes of b. The regularizer is a sorted L1 norm which penalizes the regression coefficients according to their rank: the higher the rank, the larger the penalty. This is similar to the famous BHq procedure [Benjamini and Hochberg, 1995], which compares the value of a test statistic taken from a family to a critical threshold that depends on its rank in the family. SLOPE is a convex program and we demonstrate an efficient algorithm for computing the solution. We prove that for orthogonal designs with p variables, taking lambda_i = F^{-1}(1-q_i) (F is the cdf of the errors), q_i = iq/(2p), controls the false discovery rate (FDR) for variable selection. When the design matrix is nonorthogonal there are inherent limitations on the FDR level and the power which can be obtained with model selection methods based on L1-like penalties. However, whenever the columns of the design matrix are not strongly correlated, we demonstrate empirically that it is possible to select the parameters lambda_i as to obtain FDR control at a reasonable level as long as the number of nonzero coefficients is not too large. At the same time, the procedure exhibits increased power over the lasso, which treats all coefficients equally. The paper illustrates further estimation properties of the new selection rule through comprehensive simulation studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑