arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1112.0918stat.MEstat.COstat.ML
收藏

关于最佳子集回归

On best subset regression

Shifeng Xiong

摘要:

本文讨论了\ell 0-范数约束回归中的变量选择问题,它等价于寻找一个固定大小的最佳子集问题。我们的研究集中在两个方面,一致性和计算。我们证明了在正则条件下,对于指数增长的维数,这种方法得到的稀疏估计能够渐近地保留所有重要变量。这表明最佳子集回归方法可以有效地将完整模型缩小到小于样本量的子模型,可以在随访研究中通过成熟的回归技术对此类病例进行分析。我们提供了一个迭代算法,称为正交子集选择(OSS),以解决最佳子集回归的计算问题。OSS是一种期望最大化算法,因此具有单调性。对于任何稀疏估计,OSS可以通过将其作为初始点来改善其模型的拟合。改进后的估计量保持了稀疏性。OSS的另一个吸引人的特征是,类似于连续优化问题的有效算法,如果初始点位于全局解的邻域中,OSS可以收敛到\ell 0-范数约束回归问题的全局解。本文还研究了OSS的加速算法及其与前向逐步选择的结合。仿真和一个实际的例子来评估所提出的方法的性能。

英文摘要:

In this paper we discuss the variable selection method from \ell0-norm constrained regression, which is equivalent to the problem of finding the best subset of a fixed size. Our study focuses on two aspects, consistency and computation. We prove that the sparse estimator from such a method can retain all of the important variables asymptotically for even exponentially growing dimensionality under regularity conditions. This indicates that the best subset regression method can efficiently shrink the full model down to a submodel of a size less than the sample size, which can be analyzed by well-developed regression techniques for such cases in a follow-up study. We provide an iterative algorithm, called orthogonalizing subset selection (OSS), to address computational issues in best subset regression. OSS is an EM algorithm, and thus possesses the monotonicity property. For any sparse estimator, OSS can improve its fit of the model by putting it as an initial point. After this improvement, the sparsity of the estimator is kept. Another appealing feature of OSS is that, similarly to an effective algorithm for a continuous optimization problem, OSS can converge to the global solution to the \ell0-norm constrained regression problem if the initial point lies in a neighborhood of the global solution. An accelerating algorithm of OSS and its combination with forward stepwise selection are also investigated. Simulations and a real example are presented to evaluate the performances of the proposed methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑