arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1207.5367math.STstat.TH
收藏

表征$L_2$升压

Characterizing $L_2$Boosting

John Ehrlinger, Hemant Ishwaran

摘要:

我们考虑$L_2$提升,弗里德曼的通用提升算法适用于线性回归下$L_2$损失的一个特殊情况。我们研究了任意正则化参数的$L_2$ Boosting,并导出了沿固定坐标方向的步数的精确闭合形式表达式。这个关系被用来描述$L_2$ Boosting的解决方案路径,描述研究其路径的新工具,并表征算法的一些独特属性,包括活动集循环,当正则化参数任意小时,算法在相同坐标之间花费很长时间循环的属性。我们的固定下降分析还揭示了一个可抑制的条件,通过阻止期望的变量进入解决方案路径,限制了相关问题中$L_2$ Boosting的有效性。作为一个简单的补救措施,使用类似于弹性网络的数据增强方法来引入$L_2$ -惩罚,并结合去相关,从而克服了$L_2$ Boosting在相关问题中的不足。就其本身而言,这为为什么弹性网络在相关问题中取得成功以及为什么像LAR和lasso这样的方法在这种情况下表现不佳提供了一个新的解释。

英文摘要:

We consider $L_2$Boosting, a special case of Friedman's generic boosting algorithm applied to linear regression under $L_2$-loss. We study $L_2$Boosting for an arbitrary regularization parameter and derive an exact closed form expression for the number of steps taken along a fixed coordinate direction. This relationship is used to describe $L_2$Boosting's solution path, to describe new tools for studying its path, and to characterize some of the algorithm's unique properties, including active set cycling, a property where the algorithm spends lengthy periods of time cycling between the same coordinates when the regularization parameter is arbitrarily small. Our fixed descent analysis also reveals a repressible condition that limits the effectiveness of $L_2$Boosting in correlated problems by preventing desirable variables from entering the solution path. As a simple remedy, a data augmentation method similar to that used for the elastic net is used to introduce $L_2$-penalization and is shown, in combination with decorrelation, to reverse the repressible condition and circumvents $L_2$Boosting's deficiencies in correlated problems. In itself, this presents a new explanation for why the elastic net is successful in correlated problems and why methods like LAR and lasso can perform poorly in such settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑