下采样Newton方法的收敛速度
Convergence rates of sub-sampled Newton methods
摘要：
考虑凸参数集$\mathcal{C} \subset \mathbb{R}^p$上的$n$函数之和极小化问题，其中$n\gg p\gg 1$ .在这种情况下，已知利用子采样技术的算法是有效的。在本文中，我们使用子采样技术与低秩近似设计一个新的随机批处理算法，具有可比的收敛速度牛顿的方法，但有更小的每次迭代的成本。该算法在初始点和步长方面具有鲁棒性，并且具有复合收敛速度，即初始点二次收敛，最小点接近最小点时线性收敛.我们发展其理论分析，这也使我们能够选择接近最佳的算法参数。我们的理论结果可以用来获得以前提出的基于子采样算法的收敛速度。我们展示了我们的结果如何适用于众所周知的机器学习问题。最后，我们评估了我们的算法在不同场景下的几个数据集上的性能。
英文摘要：
We consider the problem of minimizing a sum of $n$ functions over a convex parameter set $\mathcal{C} \subset \mathbb{R}^p$ where $n\gg p\gg 1$. In this regime, algorithms which utilize sub-sampling techniques are known to be effective. In this paper, we use sub-sampling techniques together with low-rank approximation to design a new randomized batch algorithm which possesses comparable convergence rate to Newton's method, yet has much smaller per-iteration cost. The proposed algorithm is robust in terms of starting point and step size, and enjoys a composite convergence rate, namely, quadratic convergence at start and linear convergence when the iterate is close to the minimizer. We develop its theoretical analysis which also allows us to select near-optimal algorithm parameters. Our theoretical results can be used to obtain convergence rates of previously proposed sub-sampling based algorithms as well. We demonstrate how our results apply to well-known machine learning problems. Lastly, we evaluate the performance of our algorithm on several datasets under various scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
