数据拟合的混合确定性-随机方法
Hybrid Deterministic-Stochastic Methods for Data Fitting
AI总结：
提出一种混合增量梯度算法，通过控制样本大小实现全梯度方法的稳定收敛率，并基于拟牛顿法给出实用实现。
摘要：
许多结构化数据拟合应用需要解决涉及大量测量值之和的优化问题。增量梯度算法通过对和中的项进行子集采样，提供了廉价的迭代。这些方法初始进展很快，但接近解时往往变慢。相比之下，全梯度方法以每次迭代评估完整目标和梯度为代价，实现稳定收敛。我们探索了兼具两者优点的混合方法。收敛速率分析表明，通过控制增量梯度算法中的样本大小，可以保持全梯度方法的稳定收敛速率。我们详细介绍了基于该方法的实用拟牛顿实现。数值实验说明了其潜在优势。
英文摘要：
Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great progress initially, but often slow as they approach a solution. In contrast, full-gradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration. We explore hybrid methods that exhibit the benefits of both approaches. Rate-of-convergence analysis shows that by controlling the sample size in an incremental gradient algorithm, it is possible to maintain the steady convergence rates of full-gradient methods. We detail a practical quasi-Newton implementation based on this approach. Numerical experiments illustrate its potential benefits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
