arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1307.2307stat.MLcs.LG
收藏

模型选择的桥接信息准则与参数收缩

Bridging Information Criteria and Parameter Shrinkage for Model Selection

Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Hong Kong Baptist University(香港浸会大学);Chinese University of Hong Kong(香港中文大学);University of Helsinki(赫尔辛基大学)

Kun Zhang, Heng Peng, Laiwan Chan, Aapo Hyvarinen

摘要:

基于经典信息准则(如BIC)的模型选择通常需要大量计算,但其性质已得到充分研究。另一方面,基于通过$\ell_1$型惩罚的参数收缩的模型选择在计算上是高效的。在本文中,我们试图结合他们的优势,并提出了一个简单的方法,惩罚的可能性与数据相关的$\ell_1$处罚自适应套索,并利用一个固定的惩罚参数。即使在有限样本情况下,其模型选择结果也与基于信息准则的结果近似一致,特别是在某些特殊情况下,该方法和相应的信息准则产生完全相同的模型.人们也可以考虑这种方法作为一种直接确定自适应Lasso中的惩罚参数以实现类似信息准则的模型选择的方式。作为扩展,我们将这一思想应用到复杂的模型,包括高斯混合模型和混合因子分析,其模型选择是传统上很难做到的,通过采用适当的惩罚,我们提供了连续逼近相应的信息标准,这是很容易优化,使有效的模型选择。

英文摘要:

Model selection based on classical information criteria, such as BIC, is generally computationally demanding, but its properties are well studied. On the other hand, model selection based on parameter shrinkage by $\ell_1$-type penalties is computationally efficient. In this paper we make an attempt to combine their strengths, and propose a simple approach that penalizes the likelihood with data-dependent $\ell_1$ penalties as in adaptive Lasso and exploits a fixed penalization parameter. Even for finite samples, its model selection results approximately coincide with those based on information criteria; in particular, we show that in some special cases, this approach and the corresponding information criterion produce exactly the same model. One can also consider this approach as a way to directly determine the penalization parameter in adaptive Lasso to achieve information criteria-like model selection. As extensions, we apply this idea to complex models including Gaussian mixture model and mixture of factor analyzers, whose model selection is traditionally difficult to do; by adopting suitable penalties, we provide continuous approximators to the corresponding information criteria, which are easy to optimize and enable efficient model selection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑