随机缩放加速草图投影牛顿方法的推断
Inference for Newton Methods with Accelerated Sketch-and-Project via Random Scaling
- Georgia Institute of Technology（佐治亚理工学院）
- Princeton University（普林斯顿大学）
- University of Michigan（密歇根大学）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
本研究提出一种基于广义加速草图投影求解器的在线牛顿方法，建立其平均迭代的渐近正态性与函数中心极限定理，并开发基于随机缩放的在线推断程序，数值实验验证了其优越性能。
中文摘要 AI 辅助
我们研究了一种在线草图牛顿方法，该方法通过一种最先进的草图求解器（称为广义加速草图投影求解器（GAS））在每一步近似牛顿方向，从而缓解了经典二阶方法的计算瓶颈。GAS求解器通过Nesterov动量更新实现加速收敛，优于朴素、未加速的草图投影求解器，并支持灵活的投影度量，其恰当选择可进一步降低计算成本。基于此设计，我们建立了平均草图牛顿迭代的渐近正态性，并刻画了其极限协方差矩阵。所得协方差在特定的加速参数选择下恢复为未加速草图牛顿方法的协方差，在一般情况下（按草图步数计）更快地收敛到极小极大最优协方差，并且小于加速方法最后迭代产生的协方差。最后，我们通过建立牛顿迭代的函数中心极限定理来强化这些结果，这使我们能够绕过显式协方差估计，并开发基于随机缩放的在线推断程序。具体而言，我们通过适当缩放平均迭代构造了一个枢轴检验统计量，使其极限分布不依赖于任何未知参数，从而实现渐近有效的在线推断。数值实验证明了所提出推断程序的优越性能。
英文摘要
We study an online sketched Newton method that approximates the Newton direction at each step via a state-of-the-art sketching solver, called the generalized accelerated sketch-and-project solver (GAS), thereby mitigating the computational bottleneck of classical second-order methods. The GAS solver improves upon vanilla, unaccelerated sketch-and-project solvers by achieving accelerated convergence through Nesterov momentum updates, and accommodates a flexible projection metric whose proper choice further reduces computational cost. Building on this design, we establish asymptotic normality of the averaged sketched Newton iterates and characterize their limiting covariance matrix. The resulting covariance recovers that of the unaccelerated sketched Newton method under a specific choice of acceleration parameters, converges more rapidly (in the number of sketching steps) to the minimax-optimal covariance in general, and is smaller than that of the last iterate produced by the accelerated method. Finally, we strengthen these results by establishing a functional central limit theorem for the Newton iterates, which allows us to bypass explicit covariance estimation and develop an online inference procedure based on random scaling. Specifically, we construct a pivotal test statistic by appropriately rescaling the averaged iterates, so that its limiting distribution is free of any unknown parameters, enabling asymptotically valid online inference. Numerical experiments demonstrate superior performance of the proposed inference procedure.
