arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1201.3528stat.COstat.ME
收藏

稀疏回归的路径跟踪与经验贝叶斯模型选择

Path Following and Empirical Bayes Model Selection for Sparse Regression

Hua Zhou, Artin Armagan, David B. Dunson

摘要:

近年来,针对高维回归问题提出了多种正则化方法。然而,调整参数的选择和超高维问题的计算效率仍然是令人烦恼的问题。$\ell_1$正则化的常规使用在很大程度上归因于LARS算法的计算效率,但是对于表现更好的惩罚的类似效率仍然是难以捉摸的。在这篇文章中,我们提出了一个高效的路径跟踪程序的任何凸损失函数和广泛的惩罚类的组合。从贝叶斯的角度来看,该算法迅速产生最大后验估计在不同的超参数值。为了绕过交叉验证的低效率和潜在的不稳定性,我们提出了一个经验贝叶斯过程,用于快速选择最佳模型和相应的超参数值。该方法适用于对应于回归系数的适当先验分布的任何惩罚。虽然我们主要关注广义线性模型的稀疏估计,但该方法扩展到更一般的正则化,例如重新参数化后的多项式趋势滤波。所提出的算法有效地扩展到大的$p$和/或$n$。10,000维的例子的解决方案的路径计算在一分钟内在笔记本电脑上的各种广义线性模型(GLM)。操作特性进行评估,通过模拟研究和方法被应用到几个真实的数据集。

英文摘要:

In recent years, a rich variety of regularization procedures have been proposed for high dimensional regression problems. However, tuning parameter choice and computational efficiency in ultra-high dimensional problems remain vexing issues. The routine use of $\ell_1$ regularization is largely attributable to the computational efficiency of the LARS algorithm, but similar efficiency for better behaved penalties has remained elusive. In this article, we propose a highly efficient path following procedure for combination of any convex loss function and a broad class of penalties. From a Bayesian perspective, this algorithm rapidly yields maximum a posteriori estimates at different hyper-parameter values. To bypass the inefficiency and potential instability of cross validation, we propose an empirical Bayes procedure for rapidly choosing the optimal model and corresponding hyper-parameter value. This approach applies to any penalty that corresponds to a proper prior distribution on the regression coefficients. While we mainly focus on sparse estimation of generalized linear models, the method extends to more general regularizations such as polynomial trend filtering after reparameterization. The proposed algorithm scales efficiently to large $p$ and/or $n$. Solution paths of 10,000 dimensional examples are computed within one minute on a laptop for various generalized linear models (GLM). Operating characteristics are assessed through simulation studies and the methods are applied to several real data sets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑