次Weibull噪声与Polyak-Łojasiewicz条件下的在线随机梯度方法
Online Stochastic Gradient Methods Under Sub-Weibull Noise and the Polyak-Łojasiewicz Condition
AI总结：
该研究在次Weibull梯度噪声假设下，分析了满足PL条件的在线梯度下降法和满足近端PL条件的在线近端梯度法的收敛性，给出了期望与高概率遗憾界，并通过仿真验证了理论结果。
摘要：
本文聚焦于存在随机梯度误差的在线梯度及近端梯度方法。具体而言，我们研究了当代价函数满足Polyak-Łojasiewicz（PL）不等式时在线梯度下降法的性能。我们给出了期望意义下和高概率意义下的界（逐迭代成立），其中高概率界是通过对影响梯度的误差采用次Weibull模型推导得到的。收敛结果表明，瞬时遗憾以线性速率收敛至一个误差项，该误差取决于问题的变异性以及次Weibull梯度误差的统计特性。随后，在复合代价函数满足近端PL条件的假设下，我们针对在线近端梯度方法给出了类似的收敛结果。在静态代价的情形下，当梯度误差被建模为次Weibull随机变量时，我们给出了这些方法所产生遗憾的新界。我们还提供了说明性仿真以验证上述理论结果。
英文摘要：
This paper focuses on the online gradient and proximal-gradient methods with stochastic gradient errors. In particular, we examine the performance of the online gradient descent method when the cost satisfies the Polyak-Łojasiewicz (PL) inequality. We provide bounds in expectation and in high probability (that hold iteration-wise), with the latter derived by leveraging a sub-Weibull model for the errors affecting the gradient. The convergence results show that the instantaneous regret converges linearly up to an error that depends on the variability of the problem and the statistics of the sub-Weibull gradient error. Similar convergence results are then provided for the online proximal-gradient method, under the assumption that the composite cost satisfies the proximal-PL condition. In the case of static costs, we provide new bounds for the regret incurred by these methods when the gradient errors are modeled as sub-Weibull random variables. Illustrative simulations are provided to corroborate the technical findings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
