大规模稀疏回归：基于一阶优化的分支定界
Sparse Regression at Scale: Branch-and-Bound rooted in First-Order Optimization
摘要：
我们考虑最小二乘回归问题，惩罚与组合的$\ell_{0}$和平方$\ell_{2}$惩罚函数（a.k.a. $\ell_0 \ell_2$正则化）。最近的工作表明，由此产生的估计是非常重要的，在许多高维统计设置。然而，这些估计的精确计算仍然是一个重大挑战。事实上，现代精确的方法，混合整数规划（MIP）的基础上，面临的困难时，功能的数量$p \sim 10^4$。在这项工作中，我们提出了一个新的确切的MIP框架$\ell_0\ell_2$ -正则化回归，可以扩展到$p \sim 10^7$，实现加速至少$5000$ x，相比国家的最先进的精确方法。不像最近的工作，它依赖于现代商业MIP求解器，我们设计了一个专门的非线性分支定界（BNB）框架，通过严格利用问题结构。在我们的框架中，一个关键的区别在于有效地解决节点松弛使用专门的一阶方法，基于坐标下降（CD）。我们基于CD的方法通过使用热启动、活动集和梯度筛选，有效地利用了BnB节点上的信息。此外，我们设计了一种新的方法，从原始CD解决方案，证明工程高维的对偶界限。在合成和真实高维数据集上的实验表明，我们的框架不仅比现有技术要快得多，而且还可以为现有方法无法处理的具有统计挑战性的实例提供可证明的最佳解决方案。我们通过我们的工具包L0 BnB开源实现。
英文摘要：
We consider the least squares regression problem, penalized with a combination of the $\ell_{0}$ and squared $\ell_{2}$ penalty functions (a.k.a. $\ell_0 \ell_2$ regularization). Recent work shows that the resulting estimators are of key importance in many high-dimensional statistical settings. However, exact computation of these estimators remains a major challenge. Indeed, modern exact methods, based on mixed integer programming (MIP), face difficulties when the number of features $p \sim 10^4$. In this work, we present a new exact MIP framework for $\ell_0\ell_2$-regularized regression that can scale to $p \sim 10^7$, achieving speedups of at least $5000$x, compared to state-of-the-art exact methods. Unlike recent work, which relies on modern commercial MIP solvers, we design a specialized nonlinear branch-and-bound (BnB) framework, by critically exploiting the problem structure. A key distinguishing component in our framework lies in efficiently solving the node relaxations using a specialized first-order method, based on coordinate descent (CD). Our CD-based method effectively leverages information across the BnB nodes, through using warm starts, active sets, and gradient screening. In addition, we design a novel method for obtaining dual bounds from primal CD solutions, which certifiably works in high dimensions. Experiments on synthetic and real high-dimensional datasets demonstrate that our framework is not only significantly faster than the state of the art, but can also deliver certifiably optimal solutions to statistically challenging instances that cannot be handled with existing methods. We open source the implementation through our toolkit L0BnB.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
