加权抽样的分批随机梯度下降法
Batched Stochastic Gradient Descent with Weighted Sampling
摘要：
我们分析了随机梯度下降（SGD）的一个批处理的变体与加权抽样分布光滑和非光滑目标函数。我们表明，通过分配的批次计算，显着加速的收敛速度是可以证明的，无论是分批采样或加权采样单独相比。我们提出了几个计算效率高的计划，以近似的最佳权重，并计算建议的采样分布明确的最小二乘和铰链损失问题。我们的分析和实验表明，可以获得实质性的收益。
英文摘要：
We analyze a batched variant of Stochastic Gradient Descent (SGD) with weighted sampling distribution for smooth and non-smooth objective functions. We show that by distributing the batches computationally, a significant speedup in the convergence rate is provably possible compared to either batched sampling or weighted sampling alone. We propose several computationally efficient schemes to approximate the optimal weights, and compute proposed sampling distributions explicitly for the least squares and hinge loss problems. We show both analytically and experimentally that substantial gains can be obtained.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
