第一千一百零六章变异绞喉
The Variational Garrote
摘要：
在本文中，我们提出了一个新的变分方法稀疏回归$L_0$正则化。变分参数出现在近似模型中的方式是类似于Breiman的Garrote模型。我们把这种方法称为变分绞喉法（VG）。我们表明，变分逼近和$L_0$正则化的组合具有的效果，使问题有效的最大秩，即使当样本的数量是小的变量的数量相比。将VG方法与Lasso方法、岭回归和最近引入的成对平均场方法（PMF）进行了数值比较。Titsias & M.拉扎罗-格雷迪拉NIPS 2012）。数值结果表明，VG和PMF产生更准确的预测和更准确地重建真实的模型比其他方法。结果表明，VG发现正确的解决方案时，拉索解决方案是不一致的，由于大的输入相关性。在全局范围内，VG比PMF快得多，并且随着问题变得更密集以及在具有强相关输入的问题中表现得更好。VG的简单实现与特征的数量成立方。通过引入拉格朗日乘子，我们得到了一个双重制定的问题，规模立方的样本数，但接近线性的功能。
英文摘要：
In this paper, we present a new variational method for sparse regression using $L_0$ regularization. The variational parameters appear in the approximate model in a way that is similar to Breiman's Garrote model. We refer to this method as the variational Garrote (VG). We show that the combination of the variational approximation and $L_0$ regularization has the effect of making the problem effectively of maximal rank even when the number of samples is small compared to the number of variables. The VG is compared numerically with the Lasso method, ridge regression and the recently introduced paired mean field method (PMF) (M. Titsias & M. Lázaro-Gredilla., NIPS 2012). Numerical results show that the VG and PMF yield more accurate predictions and more accurately reconstruct the true model than the other methods. It is shown that the VG finds correct solutions when the Lasso solution is inconsistent due to large input correlations. Globally, VG is significantly faster than PMF and tends to perform better as the problems become denser and in problems with strongly correlated inputs. The naive implementation of the VG scales cubic with the number of features. By introducing Lagrange multipliers we obtain a dual formulation of the problem that scales cubic in the number of samples, but close to linear in the number of features.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
