SEGA：通过梯度绘制减少方差
SEGA: Variance Reduction via Gradient Sketching
摘要：
我们提出了一个随机的一阶最优化方法-SEGA（SkEtched梯度法）-逐步通过其迭代建立一个方差减少估计的梯度从随机线性测量（草图）的梯度从一个预言。在每次迭代中，SEGA使用由最新草图提供的信息通过草图和投影操作更新梯度的当前估计，并且这随后用于通过随机松弛过程计算真实梯度的无偏估计。该无偏估计然后用于执行梯度步骤。与标准的子空间下降法，如坐标下降，SEGA可以用于最优化问题与不可分离的邻近项。我们提供了一个一般的收敛性分析，并证明了强凸目标的线性收敛。在坐标草图的特殊情况下，SEGA可以通过各种技术来增强，例如重要性采样，minimizing和加速，并且其速率高达与最知名的坐标下降速率相同的小常数因子。
英文摘要：
We propose a randomized first order optimization method--SEGA (SkEtched GrAdient method)-- which progressively throughout its iterations builds a variance-reduced estimate of the gradient from random linear measurements (sketches) of the gradient obtained from an oracle. In each iteration, SEGA updates the current estimate of the gradient through a sketch-and-project operation using the information provided by the latest sketch, and this is subsequently used to compute an unbiased estimate of the true gradient through a random relaxation procedure. This unbiased estimate is then used to perform a gradient step. Unlike standard subspace descent methods, such as coordinate descent, SEGA can be used for optimization problems with a non-separable proximal term. We provide a general convergence analysis and prove linear convergence for strongly convex objectives. In the special case of coordinate sketches, SEGA can be enhanced with various techniques such as importance sampling, minibatching and acceleration, and its rate is up to a small constant factor identical to the best-known rate of coordinate descent.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
