论 Newton Screening
On Newton Screening
AI总结：
本文提出内置筛选机制的广义 Newton 方法 NS 及其热启动序列版本 SNS，用于 Lasso 优化，并证明其一步局部收敛与高概率估计误差界，实验显示其在速度和精度上优于现有方法。
摘要：
筛选和工作集技术是缩小优化问题规模的重要方法。它们已被广泛用于加速求解大规模稀疏学习问题的一阶方法。本文提出了一种称为 Newton screening（NS）的新型筛选方法，它是一种内置筛选机制的广义 Newton 方法。我们推导了 Lasso 的等价 KKT 系统，并利用广义 Newton 方法求解 KKT 方程。基于该 KKT 系统，首先利用上一轮迭代产生的原始变量和对偶变量之和确定一个规模相对较小的内置工作集，然后通过在该工作集上求解一个最小二乘问题来更新原始变量，并基于闭式表达式更新对偶变量。此外，我们考虑了带热启动策略的序列版 Newton screening（SNS）。我们证明 NS 具有最优收敛性质，即它能够实现一步局部收敛。在特征矩阵满足某些正则性条件的情况下，我们证明 SNS 能以高概率得到与潜在真实目标符号相同的解，并达到尖锐的估计误差界。仿真研究和真实数据分析支持了我们的理论结果，并表明在比较研究中，SNS 比若干最先进方法更快、更准确。
英文摘要：
Screening and working set techniques are important approaches to reducing the size of an optimization problem. They have been widely used in accelerating first-order methods for solving large-scale sparse learning problems. In this paper, we develop a new screening method called Newton screening (NS) which is a generalized Newton method with a built-in screening mechanism. We derive an equivalent KKT system for the Lasso and utilize a generalized Newton method to solve the KKT equations. Based on this KKT system, a built-in working set with a relatively small size is first determined using the sum of primal and dual variables generated from the previous iteration, then the primal variable is updated by solving a least-squares problem on the working set and the dual variable updated based on a closed-form expression. Moreover, we consider a sequential version of Newton screening (SNS) with a warm-start strategy. We show that NS possesses an optimal convergence property in the sense that it achieves one-step local convergence. Under certain regularity conditions on the feature matrix, we show that SNS hits a solution with the same signs as the underlying true target and achieves a sharp estimation error bound with high probability. Simulation studies and real data analysis support our theoretical results and demonstrate that SNS is faster and more accurate than several state-of-the-art methods in our comparative studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
