arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1507.03133stat.MEmath.OCstat.COstat.ML
收藏

通过现代优化镜头选择最佳子集

Best Subset Selection via a Modern Optimization Lens

Dimitris Bertsimas, Angela King, Rahul Mazumder

摘要:

在过去的25年里(1990-2014),整数优化算法的进步与硬件的改进相结合,在解决混合随机优化(MIO)问题时,导致了惊人的2000亿因子加速。我们提出了一种MIO方法来解决经典的最佳子集选择问题的选择$k$的$p$功能在线性回归$n$的意见。我们开发了现代一阶连续优化方法的离散扩展,以找到高质量的可行解,我们使用它作为MIO求解器的热启动,找到可证明的最优解。由此产生的算法(a)提供了一个解决方案,保证其次优性,即使我们提前终止算法,(b)可以容纳侧约束的系数的线性回归和(c)扩展到寻找最佳子集的解决方案的最小绝对偏差损失函数。使用各种各样的合成和真实的数据集,我们证明了我们的方法解决问题的$n$在1000年代和$p$在100年代在几分钟内可证明的最优性,并找到接近最优的解决方案的$n$在100年代和$p$在1000年代在几分钟内。我们还建立通过数值实验,MIO方法比{ \texttt {Lasso}}和其他常用的稀疏学习程序,在实现稀疏的解决方案,具有良好的预测能力。

英文摘要:

In the last twenty-five years (1990-2014), algorithmic advances in integer optimization combined with hardware improvements have resulted in an astonishing 200 billion factor speedup in solving Mixed Integer Optimization (MIO) problems. We present a MIO approach for solving the classical best subset selection problem of choosing $k$ out of $p$ features in linear regression given $n$ observations. We develop a discrete extension of modern first order continuous optimization methods to find high quality feasible solutions that we use as warm starts to a MIO solver that finds provably optimal solutions. The resulting algorithm (a) provides a solution with a guarantee on its suboptimality even if we terminate the algorithm early, (b) can accommodate side constraints on the coefficients of the linear regression and (c) extends to finding best subset solutions for the least absolute deviation loss function. Using a wide variety of synthetic and real datasets, we demonstrate that our approach solves problems with $n$ in the 1000s and $p$ in the 100s in minutes to provable optimality, and finds near optimal solutions for $n$ in the 100s and $p$ in the 1000s in minutes. We also establish via numerical experiments that the MIO approach performs better than {\texttt {Lasso}} and other popularly used sparse learning procedures, in terms of achieving sparse solutions with good predictive power.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑